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ABSTRACT 

This publication summarizes presentations and 
discussions from a conference that addressed specific issues in the 
National Assessment of Vocational Education and in- the 1990 Perkins 
Act that mandated the assessment. Introductory materials include a 
summary of consensus issues from the conference (David Boesel) , 
welcoming remarks (Bruno Manno) , and an outline of the conference's 
purpose (Boesel) . Summaries of six panels (presentations said 
discussion) follow. "Panel 1: Legislative Intent 11 highlights the 
purposes of the legislation. "Panel 2: Effects of the Perkins Act on 
Policy and Practice" summarizes five presentations on state 
administration of vocational education programs, local practices, 
secondary and postsecondary articulation, and postsecondary policy 
and practices. "Panel 3: Funding Issues" summarizes four papers on 
expenditures for program improvement, funds allocation, funding 
formulas, and the postsecondary level. "Panel 4: Student Outcomes" 
includes seven presentations on assessing academic outcomes, success 
of the school-to-work transition, relevance of vocational training to 
subsequent employment, occupations and earnings, and employer 
satisfaction. "Panel 5: General and Special Programs" summarizes five 
presentations on correctional institutions, tribal institutions, 
flexibility of delivery systems, and access. "Panel 6: Academic and 
Vocational Issues" summarizes five presentations on teacher 
preparation, the effect of accountability on service delivery, the 
extent and success of integration of academic and vocational 
curricula, and the effect of educational reform on vocational 
education, (YLB) 
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This publication includes a summary of the presentations and discussions proceeding from the National 
Assessment of Vocational Education Design Conference, held on March 13-15, 1991 in Washington, D.C. 
The Design Conference, sponsored by OERI, was intended to address specific issues in the assessment 
and in the 1990 Perkins Act, which mandated the assessment. 

The papers that were presented at the conference appear in a companion publication, Papers Presented at 
the Design Conference for the National Assessment of Vocational Education. 

Subsequent reports will set forth the findings of the assessment. 

The National Assessment of Vocational Education was mandated by the Cari D. Perkins Vocational and 
Applied Technology Act of 1990. The conduct of the National Assessment of Vocational Education and 
the preparation of this report were sponsored by the U.S. Department of Education, Office of Educational 
Research and Improvement as mandated in Section 403(a)(1) of the Act. Any opinions, findings, conclu- 
sions, or recommendations expressed in this publication are those of the authors and do not necessarily re- 
flect the opinions of the U.S. Department of Education. 
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Summary of Major Research Questions and Issues 



David Boesel, Director 
National Assessment of Vocational Education 

The Design Conference for the National Assessment of Vocational Education resulted in 
greater consensus and fewer differences than had been expected. Some of the consensus issues and 
their conclusions are summarized below. 

First, there was consensus on how to approach a study such as this; that is, you must have 
some model in mind. You must have a concept of the organization of vocational education, of where 
it fits into the larger social and economic system, and of the directions in which many people feel it 
should go. The 1990 Perkins legislation reflects many of these elements. 

Clearly, the United States is operating in competition with other countries in the world 
economic market. That is one cause of major changes in the domestic labor market and the economy, 
including the decline of manufacturing jobs. Also, in many industries, competition is bringing about 
a broadening of the concept of an occupation, of the tasks related to a particular occupation, and of 
the skills required to carry out those tasks. 

In a related way, and perhaps consequently, we are moving toward a broader concept of 
vocational education, both in terms of the specific kinds of courses taught and the relationship 
between vocational and academic education. This relationship includes the effort to upgrade 
vocational education and emphasis on tech-prep "2+2" and "4+2" programs. The broader concept 
of vocational education is also reflected in the legislation's emphasis on "all aspects of the industry." 
These innovations engender what has been called "technological literacy." 

While we are seeing greater emphasis on broadening, upgrading, and integrating vocational 
education, we do not want to lose sight of the specific vocational skills that must be learned in this 
educational process. 

Second, there was consensus on what is emphasized in the new Perkins legislation— both 
generally and with respect to the assessment; that is, on program improvement and service to special 
populations. Program improvement is to be achieved through such initiatives as integration of 
academic and vocational education and tech-prep, among others. Improved access for special 
populations to programs of high quality is related to new funding mechanisms. 

Previous Perkins legislation included set-asides for special populations and these have been 
eliminated. Understandably, the consequences of their elimination has caused concern in both the 
Congress and the general public. The consequences of concentrating funds for students with 
handicaps and learning disabilities in specific areas, based on levels of poverty, also creates concern. 
This is an important area for the assessment to examine. As suggested at the conference, input from 
special populations should be included in the assessment. 



Educational reform efforts may also affect special populations. Higher performance standards 
may raise equity questions for those students who will find it difficult to meet the new standards. 

Past assessments have not included the education of persons in tribal institutions and 
correctional facilities, but the new assessment will look at these populations. They cannot receive 
heavy emphasis and will probably be addressed through limited surveys and case studies. 

Third, there was consensus about the amount and kinds of work that will be included in the 
assessement, based on schedules and time limitations established in the legislation. It appears that we 
will be able to look at and say some valid (though not definitive) things about the distribution of 
funds. Available information will allow us to tell what happens to Perkins dollars in schools and 
programs, though not in great detail. 

In the time available, we will be able to do process and formative studies and evaluation. We 
can look at some of the immediate state and local responses to the Perkins legislation in terms of the 
following: 1) Are standards being established? 2) Is there participation in local planning? 3) Are 
committees being set up? 4) What progress is being made in planning? There are other questions as 
well. However, we probably will be able to make only a start at addressing one of the central 
questions: What is the effect of the Perkins legislation on program improvement? There will not be 
time to reach definitive conclusions about this issue. 

It will be equally difficult to talk about access of special populations to the improved 
programs. We will be able to say something about the current state of their access to good programs, 
and we can probably define "good programs/ But we will be unable to say anything conclusive 
about whether the Perkins Act is improving programs and whether, among all the other factors at 
work, it is having an effect on access for special populations. 

We must be open with Congress a^ut the time limitations and let them know at the outset 
what is possible to do within the time frame and what will have to be done over the longer term. 
One thing that can and should be done is to describe the current state of vocational education. This 
can be done irrespective of the Perkins legislation time frame, the 5-year authorization cycle, and the 
need to say something about the impact of the legislation. We can describe vocational education and 
look at changes that have occurred over the past 5 to 10 years. We will want to compare the current 
state of vocational education to what existed previously. 

We will want to examine short-term Perkins outcomes, insofar as possible. In addition, we 
should embark on some longer-term studies that will make it possible to answer some of these 
questions in greater depth. This may be a subject of some debate. However, with such longer-term 
studies, we will come closer to definitive answers than we will be able to in the 3 years now 
available. 

We have agreed that, in conducting the National Assessment, we need to go beyond looking 
at plans and records and our own survey results— although they can be valuable. It is important that 
we go into school districts and schools and do case studies. We also should use the community as the 
unit of analysis so that we can get some sense of the interaction among employers, students, parents, 
and teachers. 
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This sort of evaluation means spending more money in a few sites and less in many sites, and 
this is the cause of a fundamental tension in the assessment. We would like to have broad, general 
representation and, at the same time, we would like to have depth. The discussions at this conference 
have pointed toward the direction of depth, of taking a closer look at what happens in schools; how 
students, parents, and employers feel; and getting a better sense of what is actually happening in the 
communities where the education is taking place. 

In conducting the National Assessment, we want to maintain a balance between secondary and 
pos^econdary education. Many of the questions in the legislation have been framed on the 
assumption that we will find the answers in secondary education. We have to expand that thinking to 
mike sure postsecondary education in community colleges and vocational/technical institutions is 
treated in the same consistent and systematic way as secondary education. 

We want to look at the role of performance assessment in changing schools and school 
systems. We want to look at teacher preparation in relation to the changes taking place in vocational 
education programs, the larger education community, and the economy. The changes we see in the 
labor market are being reflected in employer demands, which then are reflected in vocational 
education programs and courses. This affects the kinds of skills teachers need and impacts on their 
training and qualification process. 

Finally, while it is essential that we do the best possible job of looking at what is 
representative in vocational education, we clearly must look also at what is exemplary in order to 
better the enterprise. We probably will want to look at outliers in the other direction— some problem 
cases— to see why problems occur and how they can best be overcome. 

These are the major themes that have emerged from the conference and will help shape the 
assessment. As other issues and questions arise, they will be considered for possible incorporation 
into the new National Assessment of Vocational Education. 
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Design Conference for the 
National Assessment of Vocational Education 



Introduction 

The 1990 Amendments to the Perkins Act (Public Law 101-392) include a mandated National 
Assessment of Vocational Education Programs assisted under the Act. Two major thrusts of the 
Perkins Act are program improvement and access for special populations to vocational education 
programs of high quality. 

As part of the planning for the National Assessment of Vocational Education, the Office of 
Educational Research and Improvement (OERI) Office of Research sponsored the design conference 
held in Washington, D.C., March 13-15, 1991. Participants included education researchers, 
educators, practitioners, association leaders, representatives of business and labor, federal and state 
officials, representatives of special populations, and congressional staff. Panels addressed various 
aspects of the legislation in relation to assessment design. Each panel session included a discussion 
period featuring questions and comments from the participants. 

The conference was opened by Bruno Manno, then Deputy Assistant Secretary for Policy and 
Planning, OERI (now acting Deputy Assistant Secretary for Policy and Planning, Department of 
Education), who welcomed the participants. David Boesel, Director of the National Assessment of 
Vocational Education, outlined the purpose of the conference and served as the general chair of the 
conference. Milton Goldberg, Director of the Office of Research, made a brief statement and acted 
as chair of the Effects of Perkins on Policy and Practice panel. 

This report presents a summary of each presentation and discussion. Several panelists used 
overhead transparencies. The summaries of their presentations highlight points of the narratives 
accompanying the visual presentations. The summaries also include special focus on three questions 
posed by the Office of Research: 

1. What are the principal issues and research questions to be addressed by the 
assessment in the presenter's particular subject area? 

2. What are the principal data sources (e.g., computerized files, administrative 
records, articles, etc.) that can be used to address these issues and questions? If 
existing data sources are not adequate, what new data could reasonably be 
collected for the assessment? 

3. What problems should be anticipated in conducting research on this subject? 
How can these problems be overcome or mitigated? 

The discussion summaries include major questions and issues raised by participants. The 
statements in these summaries are syntheses of remarks made by one or more participants. 

Welcoming Remarks: Bruno Manno 

Bruno Manno welcomed the participants on behalf of Ted Sanders, then Acting Secretary of 
the U.S. Department of Education (now Superintendent of Public Instruction, Ohio), and Christopher 
Cross, then Assistant Secretary, OERI (now Executive Director, Education, The Business 
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Roundtable). He noted that one of the National Goals for Education adopted by the President and the 
state governors in Charlottesville, Virginia, in fall 1989 provides part of the context for the Perkins 
Act. Goal five states: "By the year 2000, every adult American will be literate and will possess the 
knowledge and skills necessary to compete in a global economy and exercise the rights and 
responsibilities of citizenship." The task of assessment and the purpose of this design conference are 
focused, but they should be considered within the wider context. 

The federal government's role in education is limited but important It has two dimensions: 
1) To promote national educational equity and to help poor children get a good start in school, giving 
disadvantaged and handicapped children extra help to assist them in their school years, ensuring 
access to a college education, and preparing them for jobs. 2) To undertake research and 
development activities to determine what works; to provide good information on the performance of 
students, schools, and states; and to provide assistance in replicating successful state and local 
initiatives across the country. 

The National Assessment ties into the second dimension of the federal role in that its purpose 
is to provide research and good information on what works in relation to the reauthorized Perkins 
Act. 

Purpose of the Conference: David Boesel 

David Boesel stated that the purpose of the design conference was to obtain advice on critical 
issues the assessment should address in relation to various areas of the Perkins Act. The presenters 
and other participants were asked to consider the critical issues for research; the data sources, both 
existing and new, that can be used to address those issues; and problems that can be anticipated and 
how they might be solved. 

It is important for the participants to consider the feasibility of various proposals for 
assessment, as the interim report to Congress is due at the end of December 1993, and the final 
report is due in July of 1994. 

Panel 1: Legislative Intent 

Chair: Bruno Manno, Deputy Assistant Secretary, Office of Policy and Planning, OERI 

Jack Jennings 

General Counsel for Education 

House Committee on Education and Labor 

Jack Jennings noted that vocational educators have expressed misgivings about the new 
legislation, especially with regard to the changes in federal allocation requirements. Many feel that 
the new legislation will destroy some positive aspects of vocational education programs; however, 
there is also some feeling that the changes will work to the good of programs. The 1990 amendments 
are the most important federal changes since 1963. At that time, vocational educators fought the 
provisions of the Act, but vocational education was strengthened and there was increased funding at 
the federal, state, and local levels. 



The 1990 changes reflect a different perspective on vocational education. Vocational 
education has not been a part of the education reform effort, and, in fact, has been seen as an 
impediment to academic reform. The Congress has heard from a number of sectors, including 
business and industry, and has decided that major changes must be made in vocational education. The 
amendments do make some radical changes and will affect vocational education funding. They will 
also change the country's perspective on vocational education and, it is hoped, will help integrate 
vocational and academic education. 

The assessment will provide Congress with facts regarding the changes between the new law 
and the old law and the effects of those changes. The Congress uses facts to counter or support 
strong opinions voiced by various segments of the population, and when an assessment is good, it is 
very useful in helping the Congress shape legislation. Business executives are saying that the 
country's school system must be changed to be a "world-class" one. The federal government is the 
lever for bringing about such change. 

Jo-Marie St. Martin 
General Counsel 

House Committee on Labor and Education 

The report, The Forgotten Half, observes that 50 percent of high school graduates do not go 
on to postsecondary education. The educational system needs to provide a good education for those 
who do not go on to college. Traditionally, there have been three tracks: academic, vocational, and 
general. However, students need to be prepared for the world of work, whether through an academic 
or a vocational program. 

The final assessment report is due in 1994; however, many of the provisions in the Act will 
not be implemented in time for the assessment to look at them. With such time constraints, the 
assessment will be a formidable task. It is important to have the data in one document that will be 
available to the Congress in time for the reauthorization. At the local V.vel, assessment of access for 
special populations should be relatively easy, while it will be more difficult to assess program quality. 

Discussion 

Major points: 

• Maryland has eliminated the general track, as it had come to be viewed as a program 
for special populations. With the new law's emphasis on access of special populations 
to vocational education and the lack of funding for program improvement, students 
may not want to be in the vocational program. 

Jennings: This could be a problem. The law is unusual in that it provides more 
flexibility for the school building in how it will spend the money. As long as there is 
full access for special populations, the money can be used to improve programs. 
Technological preparation (tech-prep) programs are not as restrictive and allow more 
discretion than standard vocational education. 

The poorest areas, including cities, often have been ignored. Under the new law, 
federal dollars will go to the poorest areas. Data indicate that the workforce shows 



greatest growth among minorities, immigrants, and women, many of whom live in 
cities and other poorer areas. States should see the new law as an opportunity to 
target these populations at the secondary level. 

St. Martin: Under the new law, monies are targeted based on poverty levels. 
Previously, the focus was on the eligibility of students; it is now on the program, at 
the building level. 

• There is a problem of maintaining the level of effort in the states under the new 
allocation formula, federal funds will probably be supplanted by local funds in many 
instances. An issue for assessment is how state and local areas will deal with the 
provisions for improving programs and ensuring access for special populations with 
limited funding. At the level the assessment needs to consider, the two provisions 
will require areas to supplant funds. 

Jennings: The Congress wants to see where the money it puts into vocational 
education is being spent. If it is blended with other funds to support a large number 
of small activities, it is hard to get accountability. The Perkins funds can no longer 
be used to support the vocational education infrastructure. 

• The law requires that the assessment demonstrate what does and does not work in 
relation to allocating funds, integration of academic and vocational education, 
performance standards and accountability, and many other areas. Since the time 
frame will make it difficult to measure student outcomes, will it be sufficient to 
provide descriptive information showing what people are doing as a result of the new 
law? 

Jennings: It will be hard to judge student outcomes on a short-term basis. However, 
in the particular areas, some things can be measured. For example, in alloca* on of 
funds, it will be possible to see how money has been shifted among area schx>!s, 
among areas of the states, among school districts, etcetera. In terms of integral* jn of 
academic and vocational education, the content areas and use of an integrated 
curriculum will give an idea of the extent of integration. 

Bruno Manno concluded the session, stating that the National Center for Education Statistics 
(NCES) has some existing data that can be tied into the assessment; for example, DOVE (Data on 
Vocational Education), formerly VEDS (Vocational Education Systems). 



Panel 2: Effects of the Perkins Act on Policy and Practice 

Chair: Milton Goldberg, Director, Office of Research, OERI 
Norton Grubb, Director 

National Center for Research in Vocational Education 
University of California, Berkeley 

Methodological issues in assessing the effects of the Perkins Act on states and localities. 

Norton Grubb's presentation outlined various methods the National Assessment might use in 
examining the effects of the Perkins Act, focusing on four areas of assessment. 

Effects on state administration and policy. The RAND/NCRVE (National Center for 
Research in Vocational Education) 50-state survey of secondary and postsecondary policies provides 
good baseline data. The states could be resurveyed to determine how policies have changed as a 
result of the Perkins Act. To corroborate this information, local districts should be suiveyed about 
their perceptions of the changes in state policies and practices. 

Potential problems. 1) State policies in areas other than vocational education may affect 
vocational education. 2) If states are only required to develop performance measures as of 1992, it 
may be difficult to determine how the measures are being used within the frame of the National 
Assessment. 3) Determining the effects of postsecondary vs. secondary programs. 

Effects on local policies and practices. The previous NAVE survey of local districts 
provides some baseline data on local district uses of Perkins funds. Other data are limited. The local 
districts and postsecondary institutions could be resurveyed to determine effects of the Perkins Act. 
Transcript analyses and completion data for the classes of 1986, 1987, 1990, and 1992 could be used 
also. At the postsecondary level, completion rates using institution specific (or state) data could be 
used. Classroom observations in sample schools and postsecondary institutions would also be 
valuable in assessing effects. 

Potential problem. Distinguishing funding patterns and policies at the state and 
local/classroom level. 

Effects on articulation between secondary and postsecondary programs. There are some 
limited baseline data obtained through a National Council on Occupational Education survey and the 
NCRVE work on tech-prep programs. The state-level effects on articulation could be ascertained 
through a survey of state policies. At the local level, where articulation can take many forms, 
examination of articulation in a sample of communities may be the best approach. 

Potential problems. 1) In regard to secondary-postsecondary articulation programs, there is 
wide variance. Some programs only supply information to the secondary schools on postsecondary 
course and degree options, while others provide methods to allow students to receive credits at the 
postsecondary institution for relevant courses taken in high school. 2) Determining how many 
students actually move from high school to postsecondary tech-prep programs. 3) Learning whether 
or not high school programs change; tech-prep can be a way community colleges and technical 
institutes affect secondary programs. 



Effects of vocational education on labor market outcomes. Existing studies using the 
National Longitudinal Survey, High School and Beyond data, and other surveys have provided a great 
deal of data. After extensive analyses, two areas remain unresolved: related/unrelated employment 
and dynamic effects of vocational education on the job market over time. Methods of assessment 
could include developing and testing crosswalks and other ways to determine the relatedness of 
employment to vocational education programs, conducting limited surveys of individuals, and 
exploring further use of unemployment insurance data. 

General observations. 1) The National Assessment should use the local community as the 
unit of analysis. 2) Postsecondary issues must be considered. 3) The time frame problem requires 
focusing on what can reasonably be done by 1993-94. 4) The current problem of state deficits would 
seem to indicate that this may not be the be?i atmosphere in which to make these changes. It is 
especially important to keep the effects of the Perkins Act separate from other effects. 5) The 
National Assessment should examine cases where change has taken place in vocational and other 
education and analyze the causes of the change. 

Phyllis Herriage 

Bureau of Technical and Vocational Education 
Des Moines, Iowa 

Effects of the Perkins Act on state administration of vocational education programs [Sec. 
403(b)(1)]. 

The Perkins Act reduces the amount of funds available for state administration and state 
discretionary activities. This reduction, coupled with the mandated changes for program structure, 
content, and emphasis, will cause the states to examine and redefine the role of the state board and 
staff. If changes at the program level are to be implemented effectively, a major statewide staff 
development effort will be required. A major question is: What effect will the reduction in funds for 
state activities have on the quality of local programs? 

There is no blueprint for state administration, as the states have various structures, emphases, 
and role descriptions. State plan development is also very different across the states, as are 
assessment approaches. Therefore, the local-level impact of state administration is different across the 
states. 

The Act emphasizes accountability. The state-developed systems of performance standards 
and measures which emphasize outcomes are managed at the local level. Iowa is a strong local 
control state, and is now looking at outcomes-based vocational education. Iowa is also looking at 
how the Perkins Act fits with the world-class schools initiatives. Two questions arise in relation to 
this aspect of the law: 1) Will the shift from state control to local control of processes have an 
impact on student achievement? 2) Will there be enough uniformity of results to draw conclusions 
about effectiveness of vocational education programs? 

With regard to state delivery systems in general, changes in the methods for distributing funds 
may impact those systems in a number of ways. States may need to examine the ability of the 
systems to serve some students. What changes, if any, will occur in the delivery of vocational 
education? 
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The increased emphasis on serving special populations in high-quality programs leads to the 
question: Will there be a state system to provide the type of technical assistance and monitoring to 
ensure the participation of special populations in vocational education programs? 

There are two other major questions related to state administration: 1) What will be the 
effects of changes in the level of funding and the responsibilities of the state on local programs and 
student outcomes? 2) How should the actual level of federal funding for state administration be 
determined given the need for effective planning, monitoring, accountability, and administration of 
vocational education? 

Lawrence Rosenstock 
RTVP, Rindge High School 
Cambridge, Massachusetts 

Effects of the Perkins Act on local vocational education practices [Sec. 403(b)(1)]. 

The Rindge High School is the second oldest vocational program in the United States. Many 
vocational methodologies are now being seen by other educators as new ways of thinking about 
education. Although unrecognized until now, vocational education has used team teaching, forms of 
cooperative learning, and experiential and applied learning. We address multiple 
intelligences— artistic, kinetic, and social as well as lingual and mathematical— and we use alternative 
forms of assessment, such as portfolios. 

In restructuring, we are seeing exportation of vocational methodologies and importation of 
academic content. Getting away from narrow skill content is important in light of the fact that only 
25 percent of ail recent vocational completers in Massachusetts work in a related occupation. It is 
also true that teen-agers do not know what they want for the rest of their lives—nor should they. We 
also know from the Bureau of Labor Statistics that most Americans change jobs four or five times. 
This emphasizes the need for a broader view of vocational education. 

Vocational education in M all aspects of an industry" is a platform upon which to integrate. 
The 300 members of our school faculty have an average age of 52, with only two under age 35. 
Integration in itself has no meaning for them. They have studied multidisciplinary education and 
believe in it, but they need to have something to build around. "All aspects of the industry" provides 
that focus. 

Two issues undergird the work of the assessors regarding local practices: 1) How well do 
people at the local level understand the provisions of the new Act, and how are they getting their 
information? 2) What are people at the local* level really doing? 

Principle research questions related to local practices: 

1. To what extent do local vocational programs integrate academic skills? Do they use a 
problem-posing and problem solving approach? Do they teach and reinforce both 
basic and higher order skills? 

2. To what extent do local programs provide all students with knowledge of and 
experience in all aspects of the industry the student is preparing to enter? (Includes 
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both the content skills and knowledge and the business management techniques and 
skills needed to thrive in the particular enterprise.) 

3. To what extent are there equal rates of participation and equal rates of successful 
completion in high quality programs (as described in item 1) for each of the special 
populations? 

4. What level of real participation or what roles do students, parents, teachers, and 
community members have in terms of actual decisionmaking, adequacy of access to 
information, and extent of communication among themselves? 

5. How well do the local actors understand the new provisions and to what extent have 
federal and state agencies made clear the profound changes called for by the new law? 

Data sources. Existing data sources that can be used to address these issues include states* 
local application requirements, local applications, states' procedures for participatory planning, states' 
monitoring reports, and local evaluation results. 

New data sources include interviews with administrators, parents, students, teachers, and 
advocates; analysis of curricular materials, observation of local classrooms, and examination of tests 
and .instruments used to assess student performance. 

Potential problems* Problems in conducting the research will be significant. The new 
provisions require things very different from existing practice. The lack of exemplars may make it 
difficult for those conducting the assessment to envision full local implementation. Also, some 
critical concepts in the new Act, such as "integration with academics" and "all aspects of the 
industry," may not be universally understood. This may lead to local practitioners using terminology 
in reporting practices that does not actually reflect the intent of the law. It will be important to 
construct assessment instruments that will distinguish practitioners who are only using the terminology 
of the new law from those who are truly integrating. 

Roy Peters 

Oklahoma Department of Vocational and Technical Education 
Stillwater, Oklahoma 

Effects of federal requirements regarding articulation between secondary and postsecondary 
programs [Sec. 403(b)(8)]. 

Several provisions of the Perkins Act are linked to articulation between secondary and 
postsecondary vocational education programs. The provisions include: 1) vocational education 
programs that offer coherent sequences of courses leading to job skills, 2) tech-prep education under 
Title III, and 3) integration of academic and vocational education through coherent sequences of 
courses so that students achieve both academic and occupational competencies. Eligible recipients 
may choose to spend Federal funds for tech-prep education programs, guidance and counseling, and 
apprenticeship programs, all of which are tied to sec^ndary/postsecondary articulation. 
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Principal issues for assessment in the area of articulation: 

1 . To what degree have the requirements resulted in articulated programs and 
participation in those programs? 

2. What are the outcomes and effectiveness of articulation between secondary and 
postsecondary programs? 

3. What factors underlie successfully articulated secondary/postsecondary programs? 

Data sources. Multiple data sources and methods of inquiry to address these issues are 
recommended. Case studies and surveys used together would strengthen the assessment design. 
Principal data sources include: student transcripts, state performance reports, articulation agreements 
and related documents, wage record and other electronic data bases, and existing literature. 

Potential problems. The first major concern for the assessors is the need to define 
"articulation/ 1 which institutions define in many different ways. There also may be differences in 
defining which students are vocational education completers. Some students may be better served in 
articulation programs than others, and some programs may lend themselves to articulation better than 
others. 

Other problems related to assessment include incomplete vocational education data bases in 
some states, differentiating pre-P.L. 101-392 articulation initiatives from those resulting from the 
Federal requirements, and philosophical differences between secondary and postsecondary education, 
e.g., competencies or job specific training vs. courses or degrees. 

Joseph Grimsley 

Richmond Community College 

Hamlet, North Carolina 

Effects of the Perkins Act on postsecondary policy and practice. 

From the perspective of the 2-year community college, the most positive aspect of the new 
Perkins legislation is the tech-prep program and the fact that the infusion of funds from the 
appropriations committee allows the program to be administered by the state. Tech-prep education 
has had a positive effect in Hamlet, North Carolina, since 1986, and is a vital part of the school 
reform agenda. 

There are also some negative aspects of the new legislation. In particular, the community 
colleges believe that the new law's requirement for developing a centralized system of core standards 
or competencies will take a great deal of time and money. The community colleges already have 
systems of standards in place; for example, curriculum standards, required program reviews, 
accreditation standards, and professional certification requirements, which should be taken into 
consideration. The new requirements raise the specter of government control of education. 



The annual evaluation requirements take a great deal of time away from service time. 
National assessments are not useful to community colleges, especially when they take time away from 



serving students. The question is whether the program's priority is to serve students or to satisfy 
federal requirements. 

Discussion 

Major points: 

• Although it is important to assess classroom practice, the time frame and the fact that 
effects on classroom practice are hard to measure indicate that some interim measures 
would be useful. To look at how state policy affects local implementation of 
articulation between secondary and postsecondary institutions, institutions could be 
visited in the near future and again in 2 years to see how the Perkins Act has changed 
their programs. The same would be true of integration between vocational and 
academic education. 

It may be hard to separate the effects of the Perkins Act from those resulting from 
other changes in policy and practice. 

• At the community college level, occupational training may be considered academic 
instruction. When integration is being evaluated, it will be important to be aware of 
the different ways institutions define academic, general, and occupational education. 

• Some states that have been developing standards and measures have determined that 
institutional measures can be used in the short run, while student outcome measures 
will be possible only in the long run. The National Assessment might want to 
consider this in developing its study design. 

• In looking ahead to reauthorization of the legislation, it is important to build on 
existing assessments rather than to begin a new process. If a new data base is created 
with every piece of new legislation, the assessments will not be meaningful in terms 
of trend analysis. 



Panel 3: Funding Issues 

Chair: Barry Stern, Deputy Assistant Secretary, Office of Vocational and Adult 
Education 

Gary Hoachlander 
MPR Associates, Inc. 
Berkeley, California 

Expenditures at federal, state, and local levels to address program improvement in 
vocational education [Sec. 403(b)(2)]. 

For many years, the Congress has been asking questions about what the federal money buys 
in vocational education. There has been no satisfactory answer. The federal portion is about 5 to 10 
percent of the total spent on vocational education. However, there are no clear and consistent records 
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of the amounts the states and local districts spend, and with few exceptions, their accounting is by 
objective not by program. It is probably not worth the effort to determine the exact percentage of 
total vocational education funds provided by the federal government, since it is such a small 
percentage of the total. It is important, however, that some effort be made to see where the federal 
money is going and what it is buying. 

The program improvement focus, as outlined in the law, is fairly clearly defined. It covers 
curriculum development, including H 2+2 H and other tech-prep programs, staff development, 
equipment purchases, guidance and counseling, remedial courses, supplementary services for special 
populations, apprenticeship programs, economic development, instruction in all aspects of the 
industry, and comprehensive mentor programs. It is also clear that the federal money is not to be 
spent to support regular program teachers and commonly held equipment. 

Research issues: 

1. Determining how much federal money is used to support program improvement 
activities. 

2. It will be harder to determine the amounts of state and local money used for this 
purpose. However, if it is possible to get this information, the assessment should ask 
if state and local money is being used to support activities in areas where federal 
money is used to purchase the services, or if it is being used to buy services in areas 
where there is no federal money available. The substitution issue is a troubling one, 
and it would be useful to determine the extent to which federal money substitutes for 
state and local funds so that services purchased with federal money in one place are 
purchased with state and local money in another. 

3. If it is possible to describe the purchase of activities, the assessment should get 
information on how decisions are made for particular activities, and why they were 
chosen. What information was used by the decision makers? What was the Perkins 
Act influence and who has input into the decision-making process? (This relates to 
issues of governance and participation in decision making.) 

Data sources. Federal, state, and local administrative records will probably be sufficient. In 
most states, aid for secondary and postsecondary education goes out under the general state assistance 
program, and it is a local decision as to how much goes to support vocational education. Some states 
with categorical funding for vocational education might have some information, but most of the 
information on program improvement activities is probably found at the local institutions. 

Potential problems. Finding out what the federal dollar buys at the local level may be 
difficult, as there are limits on the ability to attribute particular expenditures to the federal money. 
The new Perkins Act agenda moves away from accounting and toward a performance/outcome 
orientation. The important question to answer is: Does the federal government effort make a 
difference regarding student outcomes, job placement, and so forth? 
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Lana Muraskin 

SMB Economic Research, Inc. 

Washington, D.C. 

Effects of federal legislation giving the states flexibility in allocating funds to serve special 
populations [Sec. 403(b)(4)]. 

The 1990 Perkins Amendments are the culmination of and a departure from previous federal 
policy in relation to special populations. In 1990, the location of disadvantaged and disabled students 
has become the major element in determining the allocation of federal funds under the basic grant. 
The amendments suggest that Congress has become less concerned with tracing federal funds through 
the myriad levels of government to eligible individuals, and more concerned with using federal 
resources to spur institutional and district reform. This is most notable in the emphasis on integration 
of vocational and academic education and development of a coherent sequence of courses. 

Some of the key changes in the new amendments are based on findings of the previous 
NAVE. Under these amendments, funds are distributed to districts based on factors that should have 
the effect of directing greater resources to places with concentrations of disadvantaged students- 
Chapter I and handicapped populations at the secondary level; Pell grant and Bureau of Indian Affairs 
(BIA) grant recipients at the postsecondary level. The minimum grants are $15,000 to districts and 
$50,000 to postsecondary institutions. States must make the division of funds between secondary and 
postsecondary levels explicit and provide a rationale for the split. 

The greater flexibility in funds allocation is balanced in the legislation by greater reliance on 
state and local accountability for student performance. The legislation includes a requirement for 
creating and implementing a state and local system of core standards and measurements of 
performance, including competency gains, retention, and completion; additional training obtained; and 
job placement. The system also must include procedures for taking action when local performance 
fails to improve. The performance standards must specifically address incentives to encourage 
services to special populations and the means of integrating vocational education with individual 
education plans (IEP's). 

Research issues related to regulatory items. Several of these items in the current legislation 
provide guidance for program implementation. One is related to mechanisms for exemption to the 
Pell grant and BIA funds distribution procedure to postsecondary institutions within the state. It will 
be important to see the kinds of waivers that are granted. 

Other research issues related to regulatory items include looking at how postsecondary 
institutions interpret within- district allocation directions in relation to services for disabled, 
disadvantaged, and LEP students. 

Research issues related to special populations. Among the special populations issues that 
bear examining in state implementation are the rationales for distribution of funds among secondary 
and postsecondary recipients. Another item for research is assessment of the range of performance 
standards and indicators and the appropriateness of those systems developed for special populations. 
Another question is: How do states decide and establish the methods to target funds to places with 
concentrations of disadvantaged students, and how do they develop and implement the programs? 
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Research issues related to redistribution of unspent allocations. What happens when state 
evaluations reveal that initiatives in a district have failed to meet performance standards? Do states 
become involved in joint planning processes with localities? What happens as a result of those 
planning processes? 

Data sources. Questions related to regulatory items could be addressed by tracking state 
implementation of the program in a carefully selected small number of states. 

Some issues related to special populations were identified in the previous NAVE and should 
be reexamined in the current National Assessment to see what has and has not changed through the 
new guidance and rules process, and through state and local implementation. These issues are the 
bifurcation between program improvement and special populations, the narrow service provisions for 
disadvantaged and handicapped students that sometimes took place under the old Act, the kind of 
service provided to persons because they are economically disadvantaged, uses of program 
improvement funds, and very small grants to secondary schools. Another issue is inequities in 
distribution of total grants among districts with varying numbers of poor youth. 

\^ From the standpoint of funds distribution, the obvious question is whether federal programs 
result in greater resources to districts and institutions with highest concentrations of disadvantaged and 
disabled students. There is baseline information, and the survey should be repeated. The sampling 
frame would allow an examination of how the new Act may change distribution of funds to area 
vocational schools when they are in separate districts. 

Other questions triggered by the new Act are: How are funds concentrated within multi- 
school districts and what problems arise in spending funds? What process will schools which 
previously had little federal funding undertake to decide how new federal funds should be spent? Is it 
possible to implement the comparability provisions? Who participates in development of local 
performance standards and indicators, and what measures are adopted? Does any district ever fail to 
meet its performance goals, and if so, what do they do about it? How does this ambitious reform and 
assessment program play out at the postsecondary level? 

These research questions can be answered through a mix of surveys, case studies, and other 
methods. 

Steven Barro 

SMB Economic Research, Inc. 
Washington, D.C. 

Determining the most appropriate criteria and methods to direct federal funds to the states 
and to achieve the goals and objectives of the Perkins Act [Sec. 403(d)(1) and (2)J. 

The study of the funding formula was appended to the Perkins legislation, and its language 
clearly states the Congress' expectations for the analysis. The Perkins Act funds are given to the 
states through a statutory formula based on population and per capita income. Other aspects of the 
federal vocational education program have changed over the years; that is, goals, target groups, and, 
this year, the intrastate distribution mechanism. However, the interstate formula has essentially been 
fixed since 1963. 
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Issues in the fund distribution study can be framed at two levels. The generic issues arise no 
matter what program or formula is being considered. The specific issues pertain to the vocational 
education formula. The generic questions include the following: Does the formula distribute money 
in a manner reasonably related to the needs or problems the federal aid is intended to address? Is the 
formula equitable, treating places with similar needs equally and taking due account of relevant 
substantive or fiscal differences and circumstances among the aid recipients? Is the formula 
technically sound? Do the variables represent what they are supposed to, are the data reliable and 
timely, and are the elements combined in an appropriate mathematical manner? Does the formula 
create incentives to allocate or use funds or resources in certain ways, and are those incentives 
desirable in terms of the purposes of the underlying legislation? 

Before turning to the specific issues, it is important to take a look at the present formula. The 
population factor in the formula is usually interpreted as a rough indicator of a state's relative needs 
for vocational education services. The income-based adjustment factor is designed so that lower 
income states get more aid relative to population and higher income states get less aid. The factor is 
bounded to allow aid relative to population to vary by no more than 50 percent, which means that the 
lowest income state gets 50 percent more aid relative to its population than the highest state. This 
income factor is usually understood to be an adjustment for differences in states' fiscal capacity. 

In considering specific formula design issues, it is helpful to lay out the factors that appear in 
the present formula or would be added by the various proposals for making the formula more 
equitable or improving it in other ways. Formula factors are classified under four types of indicators: 
need, fiscal capacity, fiscal effort, and cost. 

The need factor. The population factor is not a good indicator of need, because there is no 
evidence that all states demand or want the same amount of vocational education per capita. Also, the 
recent emphasis on providing vocational education to disadvantaged and special needs students makes 
funding based on the general population especially incongruous. 

Under the new legislation, states are required to distribute aid among their school districts 
according to the number of poor and handicapped pupils and to take concentrations of disadvantaged 
children into account in distributing funds down to the school building level. These same distribution 
factors could be used on the interstate level. Another option would be to emulate the pupil-weighting 
schemes some states use to take account of unequal needs in funding their local school districts. 

An obvious question comes to mind: Shouldn't federal money be allocated on the basis of 
some measure of actual activity, such as the number of vocational education students and the number 
of courses taken? However, there are no state-by-state indicators of vocational enrollment or course- 
taking that could be used. 

The fiscal capacity factor. The key questions to be asked in this area: Should the 
adjustment for fiscal capacity be retained in the formula as it is at present? Is the present per capita 
income factor satisfactory for this purpose? If not, what alternative method of adjusting for capacity 
ought to be considered? 

Per capita income is not a good indicator of fiscal capacity. Some alternatives include gross 
state product per capita or the representative tax system index, which is developed and updated 
regularly by the Advisory Commission on Inter-governmental Relations. There is also a strong 
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argument that fiscal capacity in education should be measured by a per pupil variable. The study will 
need to include an assessment of the alternative capacity measures and an analysis of the possible 
effects of substituting them for the present income factor. 

The fiscal effort factor. This factor is supposed to indicate the fraction of a state's resources 
that is devoted to a specified function or program. It is used to reward states for devoting their 
resources to a function of interest to the federal government. The present formula does not include a 
fiscal effort factor; however, the new mandate states that it should be considered. 

Fiscal effort measures correspond to the fiscal capacity measures. Fiscal effort is the ratio of 
state/local outlay to state fiscal capacity, and could be measured by the percentage of a state gross 
product that is devoted to supporting vocational education. However, although data on total spending 
for vocational education are available, data on state and local spending are not. Therefore, the study 
can consider only the states* fiscal effort to support education in general. 

The cost factor. When costs differ among states, equal money does not buy equal resources 
or services. Therefore, it would be more equitable to make the target variable real dollars instead of 
nominal dollars in the formula. There is no cost factor in the present formula, so the question is 
whether or not some should be created and incorporated into a future formula. The study could 
consider some cost proxies and perhaps indicate the impact of not having such a factor in the present 
formula and the importance of including one. 

The issue of scale is related to cost, and the scale factor is usually justified by stating that 
small entities experience diseconomies of small-scale operation and should be compensated. The 
present formula has a scale factor guaranteeing even the smallest state one-half of 1 percent of the 
total amount of federal funds. However, if scale is relevant, it is relevant at the district, school, and 
program levels, rather than at the state level. And there is no reason to assume that a small state has 
small districts, schools, or programs. The study should consider the options of eliminating or 
modifying the small state factor and possibility introducing a more relevant scale factor. 

Methodology. This study requires formula simulations showing the distributions that would 
result from modifying the formula or replacing it with an alternative. The simulation results can be 
evaluated statistically, primarily by using descriptive statistics looking at interstate disparities and 
correlations of state allocations to various state characteristics. Multivariate analyses could be used to 
make comparisons looking at multiple factors. 

Potential Problems. The data gaps referred to previously mean that data on enrollment and 
course taking are not available as formula factors and cannot be taken into account in evaluating 
formulas. Also, the lack of data on the cost and financing of vocational education means that it is not 
possible to answer the question of how the distribution of federal money alters the overall distribution 
of resources among the states. The missing data also preclude the possibility of determining a 
sensible alternative, such as allocating federal money on a matching basis. 
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Rosemary Zins 

Community College of Rhode Island 
Lincoln, Rhode Island 

Perkins Act funding issues at the postsecondary level. 

For community colleges, there are three funding issues related to the Perkins Act: the 
external issue— getting the funds; the internal issue— spending the funds; and keeping the funds— the 
balancing act. 

The regions of the country have not distributed funds in an equitable fashion. Some eastern 
states have given little or no Perkins money to postsecondary education, while the western states 
generally have given more. With the general population and labor market trends toward an older 
group of workers, including many minorities and immigrants who need training and retraining, the 
pressure is on postsecondary education to help develop a world-class workforce. The perpetual turf 
issues often prevent secondary and postsecondary institutions from working together. Now, the 
Perkins Act is promoting cooperation between the two groups. 

With regard to special populations, many community colleges have already put a great deal of 
effort into enrolling special and nontraditional populations. The current formula may actually 
overcompensate in this area at the postsecondary level. If the National Assessment looks at equity 
issues at the secondary and postsecondary levels, it will find a great deal of variation. Sex equity is 
an issue at the secondary level, but it decreases as an issue as programs shift to tech-prep. 

There is some concern about gravitating toward a quota system and concern that institutions 
will be penalized if they do not meet specific numbers. 

Research issues: 

1 . Is the flow of funds between secondary and postsecondary institutions reflective of the 
demographic trends? Is the distribution equitable? 

2. Are all Perkins funds, not simply the tech-prep funds, being used to prepare a 
workforce that is technologically competitive in a global marketplace in accordance 
with the labor market indicators? 

3. Do the strategies for the integration of academic and vocational education include 
activities to promote curriculum development, student services resource coordination, 
and inservice staff training? 

4. Do the state and regional plans address the systemwide coordination among 
postsecondary and secondary educational institutions with respect to the power 
structures, the funding structures, and the articulation efforts? 

5. Are special populations not only being recruited but also being retained in vocational 
programs? 
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6. How do the programs funded through an allocation process differ from the programs 
funded through an RFP process? 

7. How can long-term folio wup on program participants be conducted from a practical 
standpoint once the grant project period has ended? 

Data sources. Self-report surveys can be used to determine the secondary/postsecondary split 
under the new Act. Other sources include administrative records on postsecondary credit and 
noncredit course enrollment and completion; intake surveys including reasons for enrollment; mail 
and telephone exit surveys; recruitment, enrollment, and retention data comparisons of Perkins 
students versus overall postsecondary student body data; case studies; descriptions of institutional 
long-range planning processes; fiscal analyses, including cost per student data; information on fiscal 
and programmatic coordination with the Job Training Partnership Act (JTPA), JOBS, and other 
funding sources; and the final narrative summative reports on Perkins programs, presented in a 
standardized format to allow for national comparisons. 

Potential problems. Major problems may be avoided if the assessment involves local 
practitioners from the beginning and by working closely with "2+2" tech-prep associate degree 
efforts, accessing JTPA data and sources, and assessing demographic and economic development 
data. The primary danger is developing a national assessment that is not practical to implement or is 
irrelevant to the practitioners. A successful national assessment will consider the basic constraints 
connected with the operation and administration of grant-funded programs, allow for flexibility at the 
local level, emphasize the importance of direct service delivery to students, and measure success by 
performance and competency outcomes. 

Discussion 

Major points: 

• The overall question is how financial distribution issues are related to outcomes. How 
are funds used to impact the program and the people it serves? 

• The National Assessment should give some attention to both interstate and intrastate 
distribution. The focus should be- on state implementation of the Act around the three 
themes of integration, articulation, and accountability. With the relatively small 
proportion of vocational education money that comes from the federal level, there will 
probably not be a large programmatic impact. The value of that money is that it 
provides legitimacy for the federal government to set some new directions for 
vocational education and to develop a national vision and policy. 

• When the new assessment is done, some schools included in the previous assessment 
may not be used for various reasons. There might be value in looking at schools and 
colleges that did have federal funds but now receive little or no federal money. This 
would be a naturally occurring experiment, and those programs could be asked some 
of the questions asked in the previous NAVE. 

• The previous NAVE used a nationally representative sample. It would be useful to 
resurvey those administrations to see what is happening with the funds and the 
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programs and practices. It would also be useful to look at communities or counties 
and take into consideration not only the school administration issues but also the local 
employment picture and parent and community interest and involvement. 

• Because it takes 3 to 5 years to implement a new law, studies done early in the 
process are really measuring the results of the old legislation. It is important that the 
assessment developers recognize this fact. Also, most studies focus on criticisms, 
with the end result that state legislators and other policymakers get the idea that the 
system is not working and develop a "tear-down" mentality. The new National 
Assessment should play down the criticisms and play up the improvements. 

• It is not clear how the assessment can measure the impact of the new law given the 
data gaps and limitations. However, the Congress does want information about the 
major changes in the law. It is possible to look at the old data, compare it with what 
is now occurring, and make some judgements. For example, the elimination of the 
set-asides could be studied by using enrollment data to see whether or not the money 
is getting closer to poor and disabled students. If the new law has not had that result, 
or if the result cannot be determined, the report can say so. It is also possible to 
determine if the states have made changes in their programs or in their appropriations. 

• With respect to providing information within the authorization cycle, it is important to 
help the Congress realize that meaningful answers cannot be obtained in a 3- to 5-year 
period and to encourage them to extend the study. 

• The Congress wants the assessment to measure and describe changes, and it also 
wants a report on the status of vocational education. Transcript studies for the 1990 
National Assessment will involve the years 1987-90 and will not be related to the new 
Act. But the information is useful in reporting the current status of vocational 
education. 

• Ongoing cohort analyses of the classes of 1990 and 1992 would allow the National 
Assessment to say something on a continuing basis about participation in vocational 
education. 

Panel 4: Student Outcomes 

Chair: Steffen Palko, Cross Timbers Oil Company 

Thomas G. Sticht 

Applied Behavioral and Cognitive Sciences, Inc. 
El Cajon, California 

Assessing academic outcomes for students of vocational education programs [Sec. 
403(b)(5)]. 

The academic outcomes measured by the NAEP (National Assessment of Educational 
Progress) probably miss about 80 percent of students' education. The NAEP (and other types of 
standardized tests) sustains the belief that reading is separate from knowledge. The knowledge 
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domain is tested in science, history, and other subjects. (Math is a body of knowledge treated as a 
skill.) The assessments of science, history, and so forth, are not called reading, although the student 
must read to do the assessment. 

To determine the academic outcomes of vocational education, it is first necessary to decide 
what we need to know. Vocational education students do acquire bodies of knowledge, and they must 
read to do so. Should vocational students have bodies of knowledge, such as science and 
mathematics, and should they know how to use that knowledge in producing services or products in 
the vocational field they are studying? Decisions must be made about assessing vocational education 
students in regard to reasoning in their vocational domains of knowledge and about the extent to 
which these students should be assessed on how well they can apply academic skills (reading, writing, 
and thinking) within their chosen fields. 

It is true that knowledge leads to knowledge and that a vocabulary test can be a proxy for a 
large body of knowledge. Also, it is said in relation to educational reform that if we can get hold of 
the assessment system, we can drive the curriculum. This is proving to be true with NAEP. 

Data sources. The Armed Services Vocational Aptitude Battery (ASVAB) includes data on 
such things as auto mechanics and shop, electronics, mechanical comprehension, as well as on such 
academic outcomes as general science, reading, and math. It tests technological literacy as well as 
academic knowledge. As students do better on the NAEP and approach the point where "everyone is 
average," a test such as the ASVAB can both measure items applying to the world of work and be an 
independent check of assessment systems developed within the regular education system and purported 
to be indicators of its effectiveness. 

Potential problems. There is a move to reform education by developing new assessment 
systems. The Commission on Skills in the American Work Force has designed an alternative 
education system that includes elementary and middle schools. At the secondary level (by age 16) the 
student in school or a youth center (essentially a drop-out prevention mechanism) will receive a 
Certificate of Initial Mastery. The measures for the certificates are being defined by the Secretary of 
Labor's Commission on Achieving Necessary Skills (SCANS). The National Assessment should look 
at this work and consider whether or not the assessment should include items reflecting the SCANS 
skills. 

James R. Stone 

National Center For Research in Vocational Education 
University of Minnesota 
St. Paul, Minnesota 

Success of the school-to-wotk transition [Sec. 403(b)(5)(C)]. 

Four aspects of the school -to-work transition will be addressed: 1) sources of the problem, 2) 
a contemporary vision of vocational education's role in the problem, and what it might be, 3) things 
lost in recent years that have facilitated the transition, and 4) the research issues and data sources. 

The problem of school-to-work transition in this country is highlighted when we compare 
ourselves to countries in Europe, to Japan, and other places. We are very different because of three 
kinds of problems. 1) In-school problems: The students of today are becoming increasingly different 

19 



from those of the past. There are many more of the students we have been accustomed to neglecting, 
such as minorities and others who come to school with problems the traditional models of education 
are not well equipped to handle. That model is inefficient for the emerging student population. 2) 
Of-school problems: Schools may not be teaching the right things in the right way. Students do not 
see education as relevant to their world, and do not place a high priority on education. The high 
school diploma means little to employers, as they know that schools are not teaching skills their 
employees need. 3) After-school problems: Our adolescents flounder after school. They do not 
begin to move into career paths until their mid-twenties, unlike students in European countries who 
generally move into occupations at ages 18-20. 

The traditional vision of vocational education's role in the problem is placing a student in a 
related occupation or in continued education or training in a related field. The new Perkins 
legislation includes tech-prep and "2 +2* programs designed to facilitate this transition, as well as the 
integration of academic and vocational education. A different way to look at transition is to see it in 
terms of integration between school-based and work-based learning. Continued learning skills rather 
than initial job placement should be the measures of school-to-work transition. Children learn in all 
settings, and the question is how that learning can be organized in ways that contribute to the 
attainment of skills necessary for success in the 21st century. This is the responsibility of all societal 
institutions, with schools as the coordinating body. 

Among the things we have lost that facilitated school-to-work transition are categorical 
funding for cooperative vocational education, strong local supervision, and incentives for industry to 
participate, such as the Targeted Jobs Tax Credit (TJTC). 

As a proposal for transition, we could say that 16-year-old students who have not received a 
Certificate of Initial Mastery cannot work. They must achieve the predetermined outcomes deemed 
necessary. Also, students who do not move to the next level can work only if the employers agree to 
participate in the students* education. This would involve a contractual agreement stating what the 
student will learn on the job, how it will be learned, and the extent to which that learning contributes 
to the necessary outcomes. The TJTC program should be revived for employers who participate. 

Research issues. 1) Measurements of the kind and amount of learning that results from 
employment. 2) The structure of programs designed to prepare students for movement from school to 
work. 3) The extent to which work experiences are genuinely integrated into the school work—how 
they advance the student's competency in basic academic skills. 4) The extent to which on-the-job 
learning is coordinated with in-school learning. 5) The extent to which cooperating employers display 
high performance work organization characteristics. That is, the quality of the job (some jobs or 
firms are better than others). 

Data sources. The national longitudinal studies, primarily the High School and Beyond 
(HS+B) and National Education Longitudinal Study (NELS) 1988 have some data that partially 
respond to the issues. Questions on school-to-work transition issues and program structure could be 
included in those ongoing studies. The Educational Testing Service's WORKLINK program is 
beginning to assemble a data base with questions important to transitions, and also could ask 
additional questions. Suggestions for new data collection include job quality data, measures of 
integration, information from students and employers, and information from teachers on program 
structure and operation. 
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In addition to surveys, a series of case studies identifying sites that are facilitating transition 
should be developed. Focus groups might be used. Some experimental designs, using communities 
and states willing to participate and develop new legislation and tax credits, could be implemented, 
allowing comparison between the experimental situations and those using a traditional model. 

Potential problems. Not all programs fall into the same mold. Measures to look at how 
program structure affects transitions must be developed. The teacher will be a confounding variable 
in this measure, but structural aspects of the program may compensate for teachers who are not as 
enthusiastic or dedicated as they might be. Finally, evaluation has many purposes; the hope is that 
the National Assessment will focus on program improvement. 

Discussion 

Major points: 

• The most valuable skill a vocational education program imparts is probably learning 
how to learn, or creating in the student the ability to adapt and learn in an ever- 
changing work environment. This holds some fairly significant consequences in 
relation to assessment. For example, it points to a move toward a process-oriented 
assessment, rather than only testing knowledge. This, in turn, represents a quantum 
leap in test design and assessment design and cost as we move from multiple choice 
testing of informational knowledge and enter an era of process or operational 
knowledge. 

• The statements about using the lack of acquiring a Certificate of Initial Mastery as a 
means to keep students out of jobs and in school are reminiscent of previous efforts to 
homogenize schooling. 

It is true that if the test is the only option, it will become an obstacle for some 
students. However, if it is one of a number of alternatives coordinated by the 
schools, it can be useful. Providing alternatives to traditional schools will not be 
cheap. 

• From an instructional point of view, should we provide a wide range of knowledge 
domains and hope that students move into particular areas of in-depth study? It is 
true that the more literate person is the one who knows a lot. A person can be taught 
to read a book, but if the person has no way of connecting with the content of the 
book— the body of knowledge it represents— it is difficult to learn anything from it. 
The purpose of education in the K-12 system is to build a large number of bodies of 
knowledge. Another purpose is to build common bodies of knowledge among people, 
so they can communicate with and understand each other. One problem is that we do 
not have a good model or concept of the mind, so in teaching and assessment we deal 
with manifestations of the mind rather than the mind itself. 
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Thomas Bailey 
Teachers College 
Columbia University 
New York, New York 

Relevance of vocational training to subsequent 
employment [Sec. 403(b)(5)(D)]. 

There are various alternatives that can be considered when we look at the relevance of 
vocational training to subsequent employment in terms of assessment. Are we talking about direct 
transfers of specific techniques or pieces of information? Are we referring to general abilities or 
skills, or to broad attitudes and behaviors? The way relevance is perceived— and the way it is 
measured— will vary depending on the perspective. 

To answer these questions, it is necessary to understand the purpose of vocational education, 
although it often is difficult to do so. One way of interpreting the 1990 Perkins Act is to see it as a 
way to attempt to redefine that purpose. Therefore, we will have a M moving target" as we look at 
historical data related to past relevance in comparison with the present. 

There are three models of vocational education. The first and simplest model is preparation 
for a specific occupation. The second model is more general in scope, providing broad occupational 
preparation and a basic educational foundation for non-college-bound or non-baccalaureate students, 
including community college students. This model emphasizes motivation, career exploration, and 
building connections to the labor market. The third, more radical, model essentially rejects the 
distinction between vocational and academic education for all students. This raises the question of 
how much integration we will provide to students who are explicitly headed towards selective, 4-year 
colleges and universities. It appears that the Perkins Act is attempting to move vocational education 
from model one to model two, but probably cannot be a primary vehicle for moving to the third 
model. 

The relevance question will change within the context of each of the three models. In model 
one, with preparation for specific occupations, we get into the training-related placement concept. A 
great deal of research has been done in this area, especially in the previous NAVE. The course- 
utilization rate was developed, using the courses and a crosswalk developed by the National 
Occupational Information Coordinating Committee (NOICC) to relate particular courses to particular 
jobs. Measures for fall 1985 indicated that the course utilization ratio overall averaged 50.9, with 
women's ratios about 20 points higher than men's. This means that about half of the vocational 
courses taken were relevant to the jobs. The question is whether or not this is a sufficient rate, and 
there is no answer. If skills and occupations are defined broadly, the ratio will be higher than if they 
are narrowly defined. 

This type of measure will become more irrelevant in the second model. When students are 
taught information about an entire industry, such as construction, a wide variety of courses are taken 
and course-to-job crosswalks cannot be developed. The skills and knowledge learned can also be 
transferred to a number of other occupations. In this model, broader measures are needed to 
determine relevance. One might be to look at types of generic work skills needed, perhaps those the 
SCANS is developing, and determine whether or not those skills are being taught in school courses. 



However, it would be very difficult to account for the fact that students are training for various 
occupations and to develop variations for the occupations. 

It is not clear whether or not people hold only one view about the third model. Views 
conflict as to whether we are talking about a fundamental restructuring of secondary and 
postsecondary education, or about a reform of vocational education. The Perkins Act's impact will 
probably be much stronger in bringing academic instruction into traditional vocational education than 
in applying a more contextual-based or applied learning approach to traditional academic courses. 
This model cannot be assessed, because it does not yet exist. However, the National Assessment 
could pay more attention to those programs where the model three strategies are being applied to a 
range of students. 

Research issuec. 1) The assessment must be explicit about the particular model and purpose 
of vocational education being considered when relevance is being measured. 2) More research is 
needed on why students do not enter the occupations for which they presumably trained. 3) Analyses 
of wages and unemployment rates should be done, as improved methodologies exist for measuring 
which types of programs lead to higher wages. A research strategy might be to set up experiments 
with different characteristics, evaluating the effects and isolating characteristics of the successful 
programs. (This probably cannot be done within the immediate time frame.) 4) Research should be 
done on broader applications of integration strategies and approaches outside of traditional vocational 
education. 

David Stevens 

University of Maryland, Baltimore County 
Baltimore, Maryland 

Occupations and earnings of former vocational education 
students. 

In designing the National Assessment, it will be important to consider the points made in the 
previous presentation about various models of vocational education and decide what the present model 
actually is. 

Concern has been expressed about the 3-year period between now and the time David Boesel 
first appears before Congress with a report. It is impossible at this time to predict what might happen 
in that period, so the assessment design must be sensitive and have built-in elements to respond to 
some of those unknowns. 

The principal research question. Are former vocational education students working in jobs 
that are consistent with their employment objectives and acquired competencies? If not, why? 

Other research issues: 

1. The complex definitions of "occupations" for assessment purposes. The Labor 

Department is working on a revision of the Dictionary of Occupational Titles , and 
NOICC and its state counterparts have occupational classification systems. The 
previous NAVE used those systems in crosswalks. However, the new National 
Assessment must take into account the major changes occurring in the industrial 
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environment, where occupational specificity is rapidly disappearing and crosswalks 
will be more difficult to develop. 



2. The distinction between wage rates, earnings, and compensation. Wage rates do not 
reflect the time worked or benefits, and earnings are the actual dollar amounts paid in 
wages. Compensation includes job security and benefit packages. 

3. The importance of knowing previous and concurrent employment histories of 
vocational education students, not just subsequent employment outcomes. 

4. Multiple institutional sources of vocational training. In the assessment, these various 
institutions should be treated as independent organizations. 

5. Unit(s)-of-analysis decisions will determine the ultimate "actionability" of findings. 
As the assessment design is developed, it is important to be clear about whose 
decisions might be different based on the possible assessment results. 

6. The relevance of exemplary and abysmal performance. Congress probably is not 
interested in average performance, so the assessment should focus on the outliers at 
both ends of the spectrum. 

Data sources. 1) Ongoing state data collection/analysis programs (Florida is doing especially 
creative work). 2) Available administrative data sources. The unemployment insurance wage record 
data is kept by the states. It allows tracking of the individual over time and shows employers, 
locations, time in particular jobs, changes in occupations, etc. 3) Case studies. 

Potential problems. 1) Multiple sources of measurement error. Vocational students may not 
use the skills they have learned, for reasons ranging from poor marketing skills to a downturn in the 
economy, with a loss of job possibilities. Vocational education programs cannot be held accountable 
for many of the student's post-program choices. 2) Limited time to compile and analyze pertinent 
data. 3) Confidentiality restrictions. State laws and the Buckley Amendment allow use of data for 
program evaluation purposes, with the statutory stipulation that no individual institution's or person's 
identity can be revealed. 4) Costs of data collection and analysis. This factor and the time frame 
constraint make it particularly important to rely on efforts already underway. 

Discussion 

Major points: 

• The previous NAVE revealed that at both the secondary and postsecondary levels, 
virtually all students take a vocational course at some point during their school 
careers. This makes it difficult to identify precisely the vocational education students. 
Rather than attempting to identify and focus on the individual vocational education 
student, perhaps the assessment should try to determine the effects of various rates of 
participation, different mixes of academic and vocational courses, et cetera, on various 
types of learning and labor market outcomes. 
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It is true that students still in school may be difficult to identify. However, program 
completers can be identified. The concept of "the vocational education student" is 
very strong, and the National Assessment probably will need to talk in terms of 
individuals and their degree of participation in various kinds of vocational education. 

It may be necessary to broaden the vocational education model to look at the kinds of 
academic experiences the vocational students have. In any model, the student who 
takes academic coursework in the "general track" tends to be taught by less competent 
teachers than those teaching in the college preparatory track. This student is a very 
different completer than one in a high school with a core curriculum, where an 
attempt is made to teach all students higher level math and science courses. Defining 
vocational education students only in terms of their experiences with the vocational 
teacher will not look at the serious problems in secondary education related to equity. 

It is important to look at the total experiences people have in secondary schools and 
the impact of those experiences over a longer period of time than one or two years 
after completion. Also, regarding the employment context of outcomes, it is 
important to remember that about 10 percent of male completers go into the military. 

Vocational student organizations are often asked how the students compare to others. 
Data are desperately needed. The National Educational Goals Panel is attempting to 
coordinate with SCANS, and if the National Assessment could play a part in drawing 
up a correlation, it would be extremely useful. 

Perhaps the issue of training-related jobs is not that important. As the concept of 
vocational education is broadened and we move toward its integration with academic 
education, we are trying to increase students' occupational options. Therefore, why is 
it important to know whether or not they enter jobs they have trained for, if other 
outcomes in their lives are as good or better than if they had entered the particular 
field? 

Occupational specificity as an outcome in vocational education became important in 
World War II. At that time, vocational education played a large role in helping the 
country become technologically literate. The concept of technological literacy is 
crucial. To the extent that vocational education makes people technologically literate, 
it enables them to survive in society, to become employable, and to adapt to new 
developments in technology. The National Assessment must respond to the Congress' 
mandate regarding outcomes, but the broader concept of technological literacy should 
not be overlooked. 

This concept ties in with the broader model of vocational education and the idea of 
"all aspects of the industry." Finding measures is difficult, but the ASVAB may 
provide a start, as it looks at nontraditional measurements. 
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Robert Martin 

U.S. Chamber of Commerce 

Washington, D.C. 

Employer satisfaction with and involvement in vocational education programs [Sec. 
403(b)(6)(D)]. 

The heart of the issue for employers is that vocational education should be a part of the 
economic development strategy of local communities, the states, and the nation. 

Reinstating the TJTC program has been suggested. However, this program does not reflect 
the area where jobs are being created. Employers who have used the credit are large companies with 
tax departments. Our organization does not believe the program should be authorized. The 
vocational education students are not being placed in Fortune 500 corporations, but in the 12 million 
jobs created by the other segment of business and industry. Incentives to encourage smaller 
businesses would need to overcome the perception that the paper work required is too big a burden. 

From the business perspective, there are a number of important, general questions: 

• Is there a connection at the state level between vocational education programs' vision 
and operation and the job opportunities in the state? 

• Are the vocational education goals and objectives being articulated and figured into 
the state's planning process? 

• Do vocational education directors meet regularly with the state governor and planning 
and budget staff? Does the economic development plan consider projected job 
growth? Is it based on companies relocating to the state or expansion of the existing 
base? 

• Are vocational education curriculum requirements, at both the state and local levels, 
designed to meet employer skill needs? 

• Is a statewide effort underway to enhance the basic skill foundation of technical 
employees? 

• At the local level, is there sufficient and qualified business representation on advisory 
committees? 

Research issues. Determining 1) the economic benefits to employers of hiring vocational 
education students, 2) the economic benefits to students of entering and staying in vocational 
education programs, and 3) how quickly vocational education completers recover from setbacks 
related to economic downturns and re-enter the job market. (The unemployment insurance data would 
be useful in tracking and comparing vocational and non-vocational students.) 

Data sources. The governor's state economic development plan and the mayor's and 
Chamber of Commerce's local economic plan are the primary sources for data important to 
businesses. 
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Anthony Sarnvento 

AFL-CIO Education Department 

Washington, D.C, 

Comments 

Before designing the assessment, it is important to reach an agreement on the goals of 
vocational education. And even if we say, for example, that one goal is better articulation between 
the educational system and the workplace, do we agree on which way the workplace is going? There 
may be competing trends among employers, and we must look at job quality as well as the number of 
jobs. Is there really a skills.gap; do employers believe there is a skills gap? 

If we start to examine the kinds of skills and knowledge employers look for, we can ask about 
the training provided by employers when people are on the job. Although billions of dollars are 
being spent on worker training, studies by the American Management Association, the Society for 
Human Resources, and others show that this money is being spent primarily on managers in high-tech 
occupations. Most of the front-line workers receive only orientation and safety training, and people 
in manufacturing or service industries generally are given little training to upgrade their skills. 

The report, America 's Choice, frames the problems we face in our economy and the society 
in a useful manner. The report states that, faced with the choice of high skills or low wages, business 
and industry are increasingly choosing low wages. If this is the route we take, the vocational 
education system that will serve the needs of the employers will be different from one which is trying 
to meet the needs of the small percentage of employers who are moving toward a high performance 
work organization where workers* judgement is valued. 

According to America's Choice, most workplaces are looking for workers who can follow 
instructions and they do not value higher skills that require judgment. They are still following the 
scientific management principles of the Frederick Taylor Workplace model. The labor movement 
wants people in trades and crafts, manufacturing, and service industries to have the broadest possible 
range of skills, not only for the benefit of the employee and the workplace, but also because it makes 
sense for our society. 

In looking at specific aspects of the new Perkins Act, the issue of the relationship between 
basic or prerequisite skills and higher-order or critical-thinking skills needs to be considered. We 
need a new model for integrating these two domains. We cannot separate knowledge and skills. The 
definition of literacy is "Using printed and written information to function in society, to achieve one's 
goals, and to develop one's knowledge and potential." This definition must be operational ized. 

We have heard a discussion of the "all aspects of the industry" concept. We not only need to 
help the potential carpenter understand how to market his services, keep books, get permits, etc., we 
also need to give young people an idea of how the workplace in general has evolved. Such questions 
as where the 40-hour work week came from help youth get an idea of the development of things they 
take for granted, such as work and vacation hours and health benefits. The importance of this lies in 
the fact that many times, when employers are asked about the skills they want and do not have in 
young employees, we begin to see that the employers* problem is actually with the attitude of many 
of the young people. 
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When we begin to think about integrating job and classroom experiences, we must consider 
how this can be done when employers are competing in an environment where the short-term return 
rather than long-term competitiveness is valued. How do we create situations where there are more 
learning opportunities for young people, both in and out of school? Perhaps we should consider 
practices similar to Japan and Western Europe which require employers to spend a minimum 
percentage of their total payroll cost on training their current workers. We also need to think about 
youth apprenticeship. The apprenticeship bill introduced by Representative [Richard] Gephardt and 
Senator [Sam] Nunn does not take into account the fact that placing more young people in the 
workplace does not guarantee that the job will include educational content. The youth will probably 
be placed in routine jobs which can help them develop good work habits but may lead only to the 
low-wage path. 

The vocational education assessment framework should get at the values of helping young 
people learn to think independently and frame problems creatively. These skills are needed for all 
types of careers, and for the good of society. 

Becky Dunn 
Education Services 
Russell Corporation 
Alexander City, Alabama 

Comments 

We have heard a great deal about theory, and techniques, and the Perkins Act. This 
presentation will give an idea of what one corporation is doing. The Russell Corporation is the 
nation's largest manufacturer of athletic wear, from uniforms for professional sports teams to sport 
and leisure wear for mass consumption. The corporate headquarters are in Alexandria City, 
Alabama, a community of about 17,000 people. Employees there and in several plants in various 
cities total about 13,000. 

The company has been in operation in Alexandria City since 1902, and has a long-time 
interest in education. It is taken for granted as an employer, and many residents do not grasp the 
technological advances of recent years. One strategy the corporation used to try to bridge the gap 
between education and the workplace was to take the high school teachers and teachers at the 
vocational education center on a tour of the plant, so they could use that experience in the classroom. 

On the national level, the company has a "Stay In School" advertising campaign. Ads have 
run in National Education Association journals, Fortune magazine, and a number of sports journals. 
The company is identified with athletics and wanted to convey to the nation its understanding of the 
importance of an academic background, diplomas, and degrees. 

We are accountable for our products. We could not stay in business otherwise. We receive 
the products of the school system in our workplace, and we believe that the education system should 
be more accountable. Our corporate offices have a quarterly meeting with the local school board, the 
superintendent, and the high school principals to discuss how we can work together. One result is 
that employees are allowed time with pay to attend parent/teacher conferences. The policy is working 
well, with over 600 conferences in the first school quarter. 
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The Russell Education Committee has been established and will look at the functional context 
and determine the training needed for the average hourly worker coming into the company. The 
company also has a partnership with the Adult Basic Education (ABE) program. It sponsors salaries 
and in-kind services to allow ABE to function year-round. There is also a partnership with the 
vocational training center. The company supplies machines, maintenance, and instructor training and 
offers first- choice jobs to students in the program. 

Management is becoming somewhat leaner and the number of people at the middle 
management level is decreasing. With these shifts in management practices, the Russell Company 
needs more 2-year technical graduates. It is very interested in the "2+2" and tech-prep programs. 

Last year, Alabama had a Governor's Internship in Industry program. High school teachers 
spent 6 weeks in business and industry. It was a highly successful program. Following the 6 weeks, 
teachers within districts came together and developed two or three lesson plans based on the 
workplace needs. The entire volume of plans is available to any teacher in Alabama. The Russell 
Corporation would like to see the curriculum for teacher education revised to include a provision for 
onsite experience in business and industry prior to entering the teaching force. 

The Russell Corporation is interested in multi-skilled employees who know how to make 
decisions, solve problems, and continue to learn. This can only be done through a broad-based 
curriculum and "learning how to learn." 

Discussion 

Major points: 

• The public perception of vocational education often is that it is a major training 
program for people entering occupations traditionally classified as "blue collar." 
When vocational education is associated with JTPA or welfare programs, many people 
see it as geared to "others" and not to their own children. If this is the prevailing 
thought, vocational education programs will not have the level of quality and the 
financial commitment needed. The National Assessment should look at the climate in 
which vocational education exists, including attitudes in the community. 

The question could be taken a step further to ask why our society does not value 
people who work with their hands. We give more respect to people who work with 
their heads. But it is true that people who work with their hands also need to work 
with their heads to be effective. This problem may begin to fade as the distinction 
between job-specific and general education blurs. Integration between vocational and 
academic education intended to help all students learn thinking, problem solving, and 
cooperative working skills should help destigmatize vocational education. 

A methodological question related to the issue of attitudes is that the way students and 
school staff identify the students as general, college, or vocational track is not 
captured by studying transcripts and counting the number of vocational and other 
courses. Attitude is both an outcome of schooling and an intervening variable which 
has an impact on students' future, and should be looked at along with the transcript 
data. 
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• The issue of concern about whether or not an individual enters the occupation for 

which he or she trained raises the question of how much specialized training should be 
offered. In many occupations that support our society, older workers and craftsman 
are the major part of the workforce. Will products and services in some of these 
occupations deteriorate if specialized training is not provided for young people? How 
much specialized training should we expect the employer to provide? If we are going 
to teach students generalized knowledge of the workplace, this can be done without 
using vocational education, which is expensive. 

We must keep in mind the fact that most vocational education program completers go 
to work for small businesses. Those businesses want workers who have the requisite 
job skills and can think; the issue is not head vs. hand. There may be a danger that if 
vocational education becomes too generic, we will have a generation of students who 
know a little bit about a lot of things, but have no specific skills or knowledge. 

There is nothing wrong with preparing people to enter the job market and get work 
experience, and then having them decide they want to go back to school to learn 
something else. The most difficult person to place in a job is the high school graduate 
with no specific skills. We have this in the general education system, with about 55 
percent of the school population not being prepared for work. But vocational 
education, with about 27 percent of the population, is blamed for the lack of 
preparation. 

Panel 5: General and Special Populations 

Chair: Charles Benson, National Center for Research in Vocational Education 

Osa Coffey 

Virginia Department of Correctional Education 
Richmond, Virginia 

Special considerations in assessing vocational education in correctional facilities. 

The correctional population in the United States is increasing at the rate of about 12 percent a 
year. The costs are enormous; the one million adults cost about $20,000 per person per year, and the 
juveniles about $45,000 per person per year. If vocational education can help reduce the prison 
population, it would be a very good investment. 

In terms of the Perkins Act, the correctional population is not a special population that people 
are very concerned with, and the 1 percent set-aside for offenders is controversial. But without that 
money, many states would have no funds for vocational education in correctional institutions. Those 
who design the assessment should keep in mind the fact that there is no uniform philosophy or intent 
as to why the federal government has funded vocational education for this group. 

Vocational education for offenders can be considered from several points: 1) It is an 
educational equity issue. These people have usually missed out on mainstream education, and we 
need to make it up to them. 2) The economics, as noted above. 3) In connection with point two, 
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vocational education might help reduce crime. 4) Such programs are good for inmates because they 
provide a productive oudet for the inmates' energy. 

The assessment can be helpful in producing findings that will give Congress guidelines on 
whether and how they should continue funding for this special population. The design should look at 
both process and outcomes. 

Research issues: 

Process 

1. What happens to Perkins monies in corrections? 

2. What input does corrections have in the state planning process for the Perkins Act? Is 
there a representative of corrections on the state council or planning committees, and 
if so, what difference has it made in funding? 

3. Who in corrections received the money? The definition of "criminal offender" is 
broad, including people charged with crimes they did not commit and ultimately are 
acquitted of committing. These people could be counted under the 1 percent set- 
aside. Also, in addition to prisons and jails, the definition of corrections can include 
detention centers and juvenile correction centers, as well as probationers and parolees. 
If the money is thinly spread across this broad spectrum, it will accomplish nothing. 

4. What other monies are available to and used by corrections? (For example, displaced 
homemaking and sex equity monies; if corrections can be classified as a local 
education agency [LEA], several other types of funds are available.) 

5. Distribution patterns in relation to the various administrative structures for 
correctional education. There are eight Correctional School Districts in the country 
that distribute money in various ways. Do those districts get more Federal funding 
than the other types of administrative structures? 

6. Is the 1991 Perkins Act bringing more or less money to the criminal offender special 
population? 

7. What effect is the "hold harmless" provision in the Perkins Act having? 

8. What are the variations among states in per capita expenditures of Perkins money on 
inmates? 

9. How does the correctional system use the Perkins money? 
Outcomes 

The correctional institution population is different from the general population. The 
corrections population is multi-handicapped and disadvantaged, with a dropout rate of 80 percent, an 
illiteracy rate of 60 percent, and high incidences of learning disabilities and other handicaps, 
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addiction, and abuse as children. The background unemployment and poverty rates are about 40 
percent. Even if people in this population leave prison with certificates and high-quality vocational 
education, they will have difficulty finding jobs because of their criminal records. Therefore, it 
would be very hard to measure the program's success by whether or not completers have jobs. 

Other correctional constraints that make the prison program different include available time, 
restrictions involving tools, and sudden transfers and releases. The assessment could look at the 
impact of these factors. Congress also will probably want to know what effect the 1 percent set-aside 
had on the recidivism rate. 

The major question for people in corrections is: What vocational programs have the highest 
success with what types of offenders, as measured by post-release employment, economic status, and 
reduced recidivism? Although corrections administrators do not expect a definitive answer, the 
assessment can provide information to guide their efforts. 

Data sources. It will be possible to find data on recidivism through the state departments of 
corrections. However, data on how former inmates are faring in terms of employment, staying off 
welfare, etc., will be difficult to find. Parole records in state employment commissions might yield 
some useful data. 

The Department of Education has just funded the National Correctional Education Data 
Collection Project. A draft survey instrument has been developed and will be piloted after approval 
from the Office of Management and Budget. The National Assessment should tie in with that project. 

In terms of case studies, the Correction Education Association and the Association for State 
Directors of Correction Education would be useful sources. It is also recommended that a correction 
expert be included in the overall assessment effort. 

Discussion 

Major points: 

• If there is a limited amount of resources for research on vocational education in 
correctional institutions, the major types of institutions covered should be prisons and 
juvenile correction facilities. To determine the impact vocational education can have 
on recidivism or post-release employment, it would be wise to investigate institutions 
with broad programs focused on vocational education. The issue is not to determine 
whether or not poor correctional education works, but if good correctional education 
works. (The Vienna Correctional Center in Southern Illinois is recommended.) 

• Identifying the target population may be difficult. States run literacy and school- 
completion programs for offenders, and may have limited money for vocational 
education. Therefore, they may target vocational education to people who will be 
released in the short term. The assessment will need to look at this issue. People on 
probation may be good sources of data on work transition. 

• Inmates involved with drugs present a problem. How can they be convinced that 
vocational education is valuable when they can make thousands of dollars a week 
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selling drugs? Many of the young inmates jailed for drug offenses also have an 
attitude of fatalism about whether or not they will survive when they leave prison, and 
they are not interested in making an investment in training or setting different goals 
for their lives. However, from the administrative perspective, this should not be a 
reason not to offer vocational education. 

Jerry Slater 

Salish Kootenai College 

Pablo, Montana 

Special considerations in assessing vocational education in tribal institutions* 

The new Perkins Act has eliminated many set-asides; however, the 1 percent set-aside for 
Native American programs is still in place, and educational institutions operated by the 400 tribal 
governments can apply for funds on a competitive basis. Most apply to operate programs on a 3-year 
basis. This is different from the process used with the state. Our applications must document our 
connections with the tribe's overall economic development plan. If the program is not properly 
managed in terms of finances or meeting the objectives, it is not funded again, and in cases where 
problems are particularly severe, the program can be dropped midway. 

The federal government's responsibility for providing educational services to Native 
Americans stems from the treaty agreements entered into when the reservations were established. 
The first educational services were provided in schools on the reservations by missionaries from 
various religious denominations. During the time of relocation, education for Indian people was 
provided through BIA boarding schools. Children were gone from their families for months at a time 
and were often subjected to abuse of various kinds. The major emphasis in Native American 
education was vocational education, usually in the narrowest terms. 

The federal government's education efforts for Native Americans have failed. The Native 
Americans' cultures have been changed, but not eradicated, and the people have not been assimilated. 
However, the qualities of life and practices that allowed Native Americans to live in relative harmony 
with their environments for thousands of years are gone. This has resulted in tragic social conditions 
for Native Americans. Alcoholism, unemployment, and drop-out rates are extremely high, and 
conditions are comparable to those in third-world nations. The 1 percent set-aside is serving a 
disadvantaged target population and clearly meeting the intention of the law. 

Over the years, the federal government has veered back and forth about whether and what it 
should provide in educational services. Recently, the BIA has decided that it is not responsible for 
Native American education, and that it is the states' responsibility. However, the states say that the 
reservations are federal land and are exempt from taxation, so they have no obligation and are 
justified in excluding Native Americans. Although there are federal laws prohibiting exclusion of 
Indian people in voting and holding school board positions, the states still practice such exclusion. 

Recently, Native Americans have had some successes by controlling, designing, and 
implementing their own educational programs. The 1 percent set-aside in vocational education has 
allowed access to funds to accomplish these programs. The Native Americans see educational reform 
as essential to avoiding the failures of the past. They are ambivalent about vocational education 
because of the previous attitude that Indians can only work with their hands. This has resulted in the 
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desire for a broader understanding of vocational education, which includes integration with academic 
instruction. 

The effectiveness of vocational programs in tribal institutions should be assessed in terms of a 
community unit analysis. A great deal of resource sharing takes place at the tribal level, and 
programs interact with one another; for example, day care, general assistance, welfare, and education. 
These groups together develop the overall economic development plan. 

Salish Kootenai Community College was established about 10 years ago as a result of 
coordinating vocational education and other funds. It serves over 800 students. The college buildings 
were designed and built and are maintained by the vocational education students. 

Research issues. 1) Do the nonschool programs integrate academic and vocational-technical 
education? 2) Do the nonschool programs have adequate support services, such as basic skills 
development, tutoring, counseling, career education, and GED preparation? 3) Does the program 
address the cultural differences and utilize cultural strengths? 4) Is the program staff culturally 
sensitive? 5) Is the training correlated with the tribal overall economic development plan and/or 
market surveys? 

Data sources. 1) Grant proposals should reflect instruction information such as curriculum, 
entry and exit skill levels, and staff qualifications. 2) Student files and transcripts should reflect 
student accomplishment and outcomes. 3) Annual program reports should reflect accomplishments, 
4) Specially developed self-assessment forms could be completed by grantees. 

Potential problems. 1) Considerable diversity in fund recipients; that is, BIA secondary 
schools, public schools, tribal colleges, Urban Indian organizations, and programs managed by tribal 
administrators. 2) Rural, isolated areas, and urban areas have different economic bases, 3) 
Considerable differences between school systems and urban or tribally administered systems in 
record-keeping requirements. 4) Definitions (for example, "Indian," which has 42 written definitions 
in the federal government, and "integration" of vocational and academic education). 

Perhaps the best solution to data needs is to require grant recipients to complete self- 
assessment and other information forms. Also, someone who is familiar with and has been involved 
in Indian education should be included in the assessment design process. As with the correctional 
institutions, successful programs that have impacted positively on the population should be assessed. 

Discussion 

Major points: 

• With the great diversity among the Indian populations, it will be important to try to 
find a group that represents some commonality. Two organizations—the National 
Indian Education Association and the National Council of Tribal Chairmen— are 
representative of the Indian people. There are also 24 tribal colleges in the country, 
chartered by their tribal governments and serving the people on the local reservations. 

• Language is an important issue. The definition of "limited English proficiency" 
impinges on Native Americans as well as other groups. Vocational education funds 
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are not used for language programs, although those programs are integrated with 
training programs. There is a great deal of diversity among groups regarding the 
amount of their own language they speak and understand. However, their languages 
are viewed as part of their identity, history, and heritage. 

• Over 90 percent of the Mississippi Choctaws speak their own language, and 80 
percent of the children entering school are non-English proficient. A majority of 
Navajos also speak their tribal language. When programs are evaluated in areas such 
as these, language becomes a real concern. In Mississippi, the vocational education 
programs include teaching assistant interpreters. Strong retention of tribal, non- 
written language will be a factor in how the vocational programs function. 

• For many tribes, the extent to which Perkins funds have led to program improvement 
will come down to the question of the total absence of a program versus establishment 
of a program, using Perkins funds. Due to the coordination of programs and use of 
funds from other sources such as JTPA, it may be difficult to isolate the effects of 
Perkins funds. But everything having to do with the vocational program itself— staff, 
equipment, facilities, and so forth— will have been paid for with the Perkins funds. 

• The issue of preparing qualified teachers is a major concern for Indian programs. It 
is important to look at the impact of the Perkins funds on teacher recruitment, 
preparation, and retention; staff qualifications; and availability of instruction. 

• Because of the rigorous funding standards in the RFP [Request for Proposal] process, 
including verification of a 65 percent placement rate for program completers, the 
assessment should have available data on job placement and academic outcomes, as 
well as on employer involvement. 

• Most tribes with BIA-funded schools do not fit the legal definition of an LEA and 
have no secondary vocational programs. It would be worthwhile for the assessment to 
look at the extent to which this deprives communities of secondary vocational 
education and the impact on articulation as defined by the Perkins Act. There also 
will be no integration at the secondary level. 

Comments by Panel Chair Charles Benson 

In his comments earlier, Jack Jennings seemed to say that unless the National Assessment 
shows progress toward new goals, the federal contribution will be greatly reduced, and that, in turn, 
will drive down state and local funds. When asked what results would be acceptable, Jennings replied 
that the Congress would be looking for quantitative answers. 

The problem here is that we will be implementing a new Act at a time when many states are 
in financial difficulty, and it will be hard to get good new programs in place in 2 years. Chapter I, 
Title I, was implemented too rapidly, and years were lost while people discovered that more than 
equipment was needed. The same thing could happen with this program. 

If the states feei they have to implement a large number of vocational/academic education 
programs in 2 years without sufficient funds, the programs will not be of high quality. The possible 
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result will be that the opportunity to use strong pedagogical aspects of vocational education as a 
means of reforming the secondary and college educational systems will be lost. The programs need 
time to lay a foundation for vocational and academic teachers to begin talking and working together. 
In many schools, the groups do not even know one another or have any knowledge of what the other 
is doing. A 2-year period of staff development would be very useful in some states. 

A second concern is related to Jerry Slater's comments. It is important that the National 
Assessment include representatives from the special populations. An affirmative action search should 
be undertaken to identify members of special populations who could receive research contracts. The 
National Assessment also should ensure that members of special populations are included on advisory 
committees and panels and attend planning meetings. 

Carolyn Maddy-Bernstein 

National Center for Research in Vocational Education 
University of Illinois 
Champaign, Illinois 

Effect of flexibility on the delivery of services to special populations. Effect of requirements 
regarding criteria for services to special populations and participatory planning [Sec. 
403(b)(4) and (8)J. 

The 1990 Perkins Act provides increased flexibility to identify large concentrations of 
handicapped and other special populations in the areas of greatest economic need. Some people are 
frightened at the idea of losing the set-aside, but it can be seen as an excellent way for programs to 
regroup and better serve the people Congress intends them to serve. 

The flexibility of delivery systems raises a number of issues: Will students with special needs 
be served in areas where there are not large concentrations of people, such as rural areas? Will 
people in groups traditionally difficult to serve, such as those with severe mental disabilities, be left 
out? 



Research questions: 

1 . To what extent are each of the defined populations being served? It is important to 
look at each of the subgroups separately. 

2. To what extent are special populations being served in areas of greatest economic 
need? To what extent are they being served in other geographic areas? 

3. To what extent are areas with greatest concentrations of special populations being 
served? 

4. How have states differed in their vocational education delivery systems for special 
populations? Are exemplary delivery systems emerging? Are cost-effective delivery 
models emerging? 
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5. Are funded vocational education programs receiving funds of the size, scope, and 
quality to be effective? 

6. What is the impact of the contributions of state level personnel (i.e., administrator of 
the Disability Education Act, Chapter I, and LEP programs) required to review local 
plans? 

7. What are the implications from the findings for future federal funding? 

8. What support services at the state and local levels have been added or taken away as a 
result of the'new legislation? 

Data sources. 1) State plans for vocational education. 2) Local applications for Perkins 
funding. 3) National, state, and local economic information. 4) Information from the state 
departments of education on the numbers and locations of students with disabilities and those who are 
disadvantaged. 5) Transcript data. 6) Interviews with appropriate personnel. 7) Comparative data, 
including the former NAVE findings, the National Longitudinal Transition Study of Special 
Education, and some of the ongoing state follow-up studies (e.g., Iowa, Florida). 

Potential problems. 1) Interstate differences in defining special populations (e.g., at-risk 
populations). 2) Varying service delivery systems may be difficult to compare. 3) The National 
Assessment study may be near conclusion before effects of the legislation are known at the grassroots 
level. 

Discussion 
Major points: 

• The issue of defining "special populations" is particularly important. The definition in 
the new Perkins Act still leaves it open to interpretation. The NCRVE finds that 
different states have various labels for many of the special groups; only the most 
obvious ones are standardized. 

• With regard to the community colleges* service to special populations, it will be 
necessary for the colleges to lobby to get as much of the Perkins money as possible, 
depending on the state formulas and decisions. The emphasis on tech-prep should 
help, as it requires articulation between the college and the secondary institution. 

• One effect of the new legislation may be a move toward more service to special 
populations through mainstream education, and away from sheltered programs. The 
assessment might look at the results of that movement. 
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L. Allen Phelps 
The Vocational Studies Center 
University of Wisconsin 
Madison, Wisconsin 



Participation in vocational education programs, including access of individuals who are 
members of special populations to high quality vocational education programs [Sec. 
403(b)(4)]. 

The structure of the new Perkins Act raises at least three significant questions that must be 
considered by the National Assessment. 

1. To what extent does the more flexible funding strategy enhance or limit the 

participation of special populations? What losses or gains are realized in this new 
arrangement relative to funding, interagency coordination, evaluation, data collection, 
and program improvement? 

The previous NAVE and other studies have provided some good data on access to vocational 
education for students with special needs, particularly students with disabilities. The National 
Longitudinal Transition Study, sponsored by the Office of Special Education and Rehabilitation 
Services in 1987, looked at 8,000 students with disabilities, in the age bracket 13-20. 

A major analysis of the study's findings looked at the effects of occupationally oriented 
vocational education on the special needs youth in comparison with nonparticipating students. The 
analysis showed that the participating youth had significantly lower school absenteeism and a 
significantly lower probability of dropping out when demographic and disability were controlled. 
Independent of its effects on students* decisions to drop out, vocational training was significantly 
related to a higher likelihood of finding a job and participating in vocational education at the 
postsecondary level in the early years after high school. One major factor to be considered in relation 
to the implications of this study is that only 56 percent of students with disabilities were enrolled in 
vocational education programs during the most recent high school year. 

Another general issue related to access for special education concerns the reauthorized 
Individuals with Disabilities Education Act. The new legislation revises the previous definition of 
free and appropriate public education to include transitional services, seen as an outcome-oriented 
process that promotes movement from school to post-school activities, including vocational training 
and postsecondary education. The students* IEP's must now include a statement about the transition 
services. Thus, this initiative looks beyond the specifics of vocational or special education to include 
a process that coordinates resources and programs to ensure that this special population will have a 
successful transition into the workplace. 

In relation to the question of more flexible funding and access, and the impact of this 
flexibility on program improvement and other aspects of vocational education, a number of policy 
questions should be answered: What student outcomes can be generated with limited resources under 
the new Act? What happens when total funding is reduced due to removal of excess costs and 
matching requirements? What happens when mandated services for disabled or disadvantaged 
students are replaced with assurance statements and performance standards designed "to encourage the 
service to target populations and target groups in special populations"? How will fewer dollars affect 
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interagency coordination? Will we be able to sustain the commitment to serving a significant 
proportion of disabled and disadvantaged students in area vocational centers and through 
comprehensive cooperative education and work programs? 

Relatively little is known about the personal value of vocational education to youth and adults 
with special needs. The National Assessment could consider developing a series of case studies 
describing the vocational education experiences of 75-150 representative students at various age levels 
with different kinds of disabilities and special problems. This would provide a rich data base to help 
Congress, administrators, and other policy-makers understand the implications of engaging these 
students in programs that improve their sense of self-esteem, give them a sense of purpose, and 
empower them to discover new avenues for learning. 

2. In what ways and to what extent does the upgrading of skills in vocational education 
through the tech-prep and integration of academics initiatives limit program access for 
special groups? 

The issue of upgrading vocational education programs through tech-prep and integration 
initiatives is critical. People in the field express apprehension that the move toward more 
"knowledge-intensive 11 programs will lead to a two-tiered system with special population students left 
in "regular" vocational education classes. We must be aware that as normative expectations 
for vocational education rise in our society, the disabled and disadvantaged clients* expectations will 
also rise. 

In relation to this issue, the National Assessment could look at state and local implementation 
of the higher level programs and ask: What is the level of participation of special population 
students? What are the outcomes for these students? The 1990 data from the National Longitudinal 
Transition Study might yield some relevant data. 

3. What are likely to be the effects of performance standards and measures on the equity 
goals of vocational-technical education? 

The effects of performance standards on equity goals issue is related to up-scaling vocational 
education programs. It raises the issue of "creaming." If the required standards on employment and 
learning gains are set sufficiently high to meet industry standards, participation of special needs 
students might be viewed as problematic in achieving aggregate, program, or course-level outcomes. 
Incentives for creaming are reduced when program evaluations are based on individual trainee 
performance gains. However, we have noted that the short-term performance measures in the 
National Assessment are likely to focus on programs rather than individuals, which adds immediacy 
to this problem. If there is a tendency to restrict participation of special populations in mainstream 
programs, there could be negative effects on previous gains related to mainstreaming. 

A corollary to value-added performance standards and creaming is consideration of cost- 
benefit measures related to not providing programs and services. A study by David Stern and some 
colleagues of the cost benefits of the California Peninsula Academy Program contains some pertinent 
data. 

The issue of performance standards provides an opportunity for the vocational education 
community to experiment at the local level. The needs of the various special populations in the 

39 

48 



community at the secondary and postsecondary levels can be carefully assessed and viable outcome 
measures identified (e.g., lowering the drop-out rate). Programs can then be designed around the 
special populations' needs and data can be collected on whether or not the outcomes were met. If 
they are not, programs will have a data base they can use to adjust the standards and services to 
match more appropriately the needs of special groups. 

Summary. Data-based studies need to be developed which carefully examine: access and 
participation levels in light of changing Federal requirements; the efficacy of selected vocational 
education interventions (e.g., supervised work experience) in achieving employment and future 
education outcomes; and the transition process experienced by students leaving vocational -technical 
programs, particularly at the postsecondary level. 

A series of case studies and evaluations should be undertaken that explore the involvement 
and outcomes for special needs students participating in tech-prep programs and other work-based 
learning programs. 

Paul Wechstein 

Center for Law and Education 
Washington, D.C. 

Comments 

Among the activities of the Center for Law and Education is representing the full range of 
students served by vocational education. The equity and special population issues cannot be separated 
from the quality issues. Indeed, they should be defined in terms of the quality issues. The key equity 
issue: Are the students in vocational education in a separate, inferior education track, routed to an 
inferior sector of the economy? Unless this basic quality question is dealt with, all other equity issues 
become relatively meaningless. 

In addressing the issue from an assessment and evaluation viewpoint, the local evaluation 
requirements of the new legislation are important. Local recipients must conduct an annual evaluation 
based on the standards and measures. The evaluation must include a review of programs with the full 
and informed participation of representatives of individuals who are members of special populations, 
to identify and adopt strategies to overcome any barriers that are resulting in lower rates of access or 
of success in programs. This is a developmental process which involves the special populations 
themselves. The bottom-line issue: How well are the special populations doing in terms of their 
participation and their success in quality programs? 

When the National Assessment has determined how it will assess quality, the key starting 
point for looking at the special populations issues is to disaggregate the quality data. This should be 
done in two ways. First, by looking at outcome data, not only by program but also by special 
population status within it. (The problem of the kinds of outcome data available to the National 
Assessment by the end of the study may have implications for this.) Second, after having identified 
high-quality programs for which the assessment looks at a variety of structural things related to the 
program, disaggregate the data by student enrollment to determine if special population students are 
concentrated in programs with characteristics of high-quality or low-quality programs. 
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The two things central to the definition of quality are the academic and the occupational 
aspects. On the academic outcome level, quality means achieving basic and advanced skills across the 
full range of academic subjects. The law has three programmatic objectives related to this which 
reinforce each other: 1) integration of academic and vocational education, 2) sequential course of 
study leading to academic and occupational competencies, and 3) assessment requirements that tell 
whether and to what extent programs are providing advanced academic skills in a vocational setting. 

The other side of the quality issue is the occupational aspect, which has not received as much 
attention as the academic aspect in terms of what is meant by quality. Central to quality on the 
occupational side are strong experience in and understanding of "all aspects of the industry." The 
Perkins Act includes this in the local assessment and evaluation requirements. 

The importance of focusing on all "aspects of the industry" has been discussed in previous 
presentations. In an equity context, and in light of the question of vocational education as an 
"inferior" track, the idea of "all aspects of the industry" helps break down the distinction between the 
people in the academic track who will go to college and who will think and make decisions and the 
people will not go to college and who will execute those decisions. The mission of vocational 
education is still focused on work and industry, but the "all aspects of the industry" concept cuts 
across the distinction and helps vocational students understand the full context of the workplace. 

The Act's definition of "general occupational skills" also speaks to this issue. These skills are 
not viewed as a group of work-readiness skills, but as a concrete knowledge base about a particular 
industry. The aim of the Act is for all students to achieve the basic set of competencies, knowledge, 
and skills. However, if the outcome data for special populations shows a normal curve equivalent 
gain for the special population students, it still does not show whether or not they are mastering the 
skills expected for all students. 

Although the National Assessment will not be able to determine much about the impact of the 
new Perkins Act on student outcomes, it can look at the quality of the state and local mechanisms 
designed to produce outcomes, including state standards and measures. This would not be a substitute 
for national aggregate data. The National Assessment needs to look at the actual provisions of the 
law and the state and local practices so that when something fails, it can show whether the cause is a 
provision of the Act that did not work or the fact that people at the local level did not understand that 
provision when setting up the program. 

Data on the disabled population need to be disaggregated in terms of degree of disability. We 
may find that students with mild learning disabilities are being well served but that someone with a 
more severe disability is being trained to be a cook, no matter what his/her own career preference 
might be, because only the culinary arts teacher knows how to work with that disability. Neither the 
interstate or intrastate formulas take account of concentrations of LEP students, which vary widely 
among states. 

One issue is related to set-asides. It has been noted that the set-aside for people in 
correctional institutions is a barrier in terms of other agencies assuming responsibility for them in the 
mainstream programs. However, under the law, they are a special population and should be treated 
as other special populations are treated. Looking at this in terms of women and sex equity issues, we 
find that women are protected only if they are "people participating in programs designed to eliminate 
sex bias." Therefore, a woman who has not been offered such a program is not protected under 
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special populations. The National Assessment may need to look at how states are interpreting this 
provision. 

The new Act moves away from funding fragmentation and has few statements about specific 
activities that must be funded. Instead, it includes a number of statements about what recipients must 
do as a consequence of getting the funds. This has some important implications for the assessment, 
as people in the field may have some difficulty understanding the new concept. The local application 
section states that the local recipient should describe how funds from the basic grant and from other 
sources will be used to meet needs such as integration of academic and vocational education and 
service to special populations. This will tell us something about specific ways state and local 
resources are used. 

The Congress wants vocational education to improve the basic programs it offers, in terms of 
academic content, broad occupational content, and the degree to which they equitably serve special 
populations. They are providing federal funds for that purpose, and funds from various other federal 
programs are also available to help improve service to meet the needs of special populations. 

The final issue is participatory planning, which the National Assessment is commissioned to 
evaluate. The Act calls for the state to establish effective procedures, by which parents, students, 
teachers, and area residents can participate in the decisions that affect them. This reflects the basic 
principle that when we try to improve something for a group of people, the results are much more 
likely to be positive if the people affected play a role in its design and implementation. That principle 
is axiomatic in a democratic society. 

Discussion 

Major points: 

• The issue of access for special populations at the community college level presents a 
problem. For some people, such as an LEP student or an illiterate adult, access may 
include 2 or 3 years of preparation before beginning the actual vocational education. 
These students may not be counted in the vocational system. This should be 
considered when outcomes are being measured and as programs are designed to 
satisfy provisions in the new law. 

• It is also important to look at the scope of support services for the diverse students 
coming to the community college, including women with children and adults with 
second- or third-grade reading ability. Vocational education depends on adult basic 
education to prepare students for vocational/technical training. 

• The idea of participatory planning and including parents in the process seems 
somewhat out of place for the adult population in most community colleges. It would 
also present a problem for programs in correctional institutions. The National 
Assessment would need to visit programs to see what is actually taking place, as 
opposed to written descriptions of the participatory planning process. 

The language in the legislation is based on Head Start, which has always included 
family and community participation. This participation has worked in Head Start but 
has not worked very well in other federal education programs. The Perkins 
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legislation intends for programs to have policy councils with parents included in the 
actual decisionmaking. It is not clear whether or not this is meant to apply to 
programs at the postsecondary level. 

This Act opens up the potential for more parental energy in the school. If vocational 
education becomes a stronger resource for meeting a range of community needs and 
becomes involved in community development activities such as child care and 
environmental issues, parents will have many new reasons for relating to the schools. 
Parental involvement on the decision-making level becomes a sensible vehicle for 
providing correct information on policy and programs and developing institutional 
commitment. 

When the Act talks about integration, does it mean integrating academic and 
vocational education throughout the schools or integrating academics into vocational 
education? Does the same approach apply to special populations as to other students? 



It probably means full restructuring of the educational system to integrate academic 
and vocational education. However, much depends on reauthorization of the Higher 
Education Act. While vocational programs are not named in special and 
compensatory education legislation, there is a genuine concern about outcomes after 
high school and how we can ensure that the entire American high school population is 
better prepared to live, work, and pursue postsecondary education. When the 
National Assessment looks at assessing the impact of the Perkins legislation, it should 
keep in mind that this may be a broader change agent than we had perceived. 

Although the legislation probably is geared toward complete integration, with all 
students participating in both college preparatory and vocational education, it is 
doubtful that the Act can achieve this. The NCRVE is trying to develop networks of 
teachers who are committed to the concept of fully integrated education, so that even 
if the Perkins Act does not mandate it, there will be a number of people who want to 
do it and will encourage others. It may also be true that if a school tries to implement 
all of the Perkins legislation, it will run into practical difficulties if it tries to maintain 
separate tracks. 

In Michigan, over two-thirds of the high schools do not give grades for vocational 
courses, and on the transcript, the courses are usually listed as electives. The high 
schools are sending the message that vocational education is not academic education. 
In the community colleges, however, all of the occupational programs are considered 
to be academic, along with specialty, support, and general education courses, such as 
American Government and electives needed for the associate degree. The support 
courses may be the area where integration lies, as they are the general education 
courses directly related to the specific occupation or cluster of occupations. 

Integration happens at many levels: in a class where teachers work together and/or 
academic and vocational content are combined; when schools are reorganized around 
organizational clusters instead of in regular departments; and when courses are aligned 
so that there is a full vocational education program rather than electives and various 
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programs that are not coherent. Integration is not well defined and varies a great deal 
at the secondary level. Not much is known about it at the postsecondary level. 
Perhaps the biggest issue for assessment is that the rationale for integration at the 
secondary level may not apply at the postsecondary level. 

The secondary-level rationale concerns such things as the specificity of vocational 
education in secondary programs and the need for students to have more general 
skills, whether they are going into the workplace or into college. The postsecondary 
situation is different, as many people are already focused on a career they are working 
toward. The available data suggest that people take general education courses related 
to their specific coursework, and that is the postsecondary definition of "integration/ 

• It is clear that Perkins funds are not going to solve all the problems for special 
populations, so it is important for the National Assessment to look at what the 
institution is doing to obtain and use funds from other programs and sources. 

• The presentation on Native American education indicates that many minorities have 
rich and unique educational histories and values that have developed around those 
histories. In the black community, the question of the role of academic education vis 
a vis economic development continues to be debated. It will be interesting to see if 
integration where young people are taught both a core curriculum that will lead them 
to postsecondary education and a curriculum that will lead them to a job will elicit 
stronger interest in the black community than has traditional vocational education. 

It is imperative that minority group members, as well as members of special 
populations, participate in planning at the national and local levels. The minority 
group members can bring valuable insight from their unique historical perspectives 
about why we are seeing success or non-success in particular programs. 

Panel 6: Academic and Vocational Issues 

Chair: Gene Bottoms, Southern Regional Education Board, State Vocational Education 
Consortium 

Comments 

A survey of 500 vocational teachers in 15 states and 38 school sites shows that only 6 out of 
10 teachers believes integration is important in reading, 3 of 10 in writing, and 5 of 10 in science. 
When asked if they do it well, the results were one of three, one of six, and one of five in reading, 
writing, and science respectively. With no funds in the new Perkins Act for teacher education and 
curriculum development, it is difficult to see how this gap can be closed. 

A similar survey of academic teachers shows that, in terms of importance, 6 of 10 believe 
reading and math are important in integration, but only 3 of 10 science teachers believe it is important 
to link science to utility. When asked if they do integration well, 3 of 10 in math responded 
positively and only 1 of 10 in science. 
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Curtis Finch 

Virginia Polytechnic Institute and State University 
Blacksburg, Virginia 

Preparations and qualifications of teachers of vocational and academic curricula in 
vocational education programs, as well as shortages of such teachers [Sec. 403(b)(3)]. 

Over the past decade, there has been a movement toward educational reform, including 
teacher education reform, that has had significant impact on our schools. There also has been reform 
in preparation of administrators. These reform movements are continuing, and teachers are dealing 
with different teaching content and structures and a changing environment. 

A number of issues surface in regard to work being done in this area by the NCRVE and 
others. The overriding question is: What is the optimum mix of preparation courses and 
experiences? This becomes particularly important in light of articulation and integration. 

The issues include 

• Teacher preparation to integrate academic and vocational education. 

• Teacher preparation to work with tech-prep education. 

• Continuing preparation and development of practicing teachers. 

• Alternate paths to the profession. 

• Part-time and full-time teachers. 

• Degreed and nondegreed teachers. 

The issue of alternate paths into the professions is one that parallels some notions of the 
reform movement in general education. People come into vocational teaching positions from many 
areas. We cannot take the elementary education model and apply it to vocational education, as 
vocational education is more complex. 

With regard to part-time teachers, some postsecondary institutions have 40-50 percent of 
teachers working part-time. The students are exposed to a great deal of course work presented by 
people from business and industry who have little or no teacher preparation. 

In the area of degreed and nondegreed teachers, about 70 percent of the trade and industrial 
teachers do not have bachelor's degrees. In some states, the only thing required to teach vocational 
subjects in high school is 3 years of occupational experience and a GED, with no preparation 
program. As we move into a new phase of vocational education, with integration of academic 
education, it would appear that teachers should have greater facility in academic subjects than is 
presently required in many places. The legislation's intent is not to have auto-body teachers working 
in English classes. However, it is important to look at the selection criteria and preparation for 
teachers who will be working in integrated programs. 
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In the area of qualifications, many people may be prepared, but they are not qualified to teach 
in vocational education. Since people come to vocational education from many different fields, it is 
particularly important to look closely at their qualifications in terms of integration and tech-prep 
education. It might be important for the National Assessment to look at teachers not only as 
individuals but also as team members. Integration of academic and vocational education will require 
cooperation. There may also be an issue related to teachers with a number of years of occupational 
experience among their qualifications, and how that narrow focus can contribute to an integrated 
program. 

With regard to the issue of supply and demand, there appears to be a shifting demand from 
secondary to postsecondary teachers, which might be a useful research issue. 

Summary of research issues. 1) Teacher preparation and qualifications to integrate academic 
and vocational education and to implement and maintain tech-prep programs; 2) Teacher supply and 
demand; 3) Vocational teacher occupational experience; 4) Alternate paths into the vocational 
teaching profession. 

Data sources. 1) A study for NCRVE entitled "The National Data Base on Vocational 
Teacher Education," which looks at preservice preparation of vocational teachers; 2) An Ohio State 
"National Survey of Vocational Teacher Education," which includes both preservice and inservice 
preparation; 3) An annual AACTE survey of teacher education enrollment; 4) A study completed at 
Virginia Tech last year for the NCRVE on content of the 4-year degree programs for prospective 
vocational teachers, which categorizes the courses and gives an idea of their initial readiness to work 
in integrated programs. 

Potential problems. Research problems emerge from the diversity of vocational teachers and 
teaching. The range of teacher qualifications and school settings, the variety of professional 
opportunities available to teachers, and variation in state certification requirements are among the 
factors that make the task of gathering meaningfiil data difficult. The issue is further complicated by 
the fact that in an integrated program, even teachers of academic subjects in a sense become 
vocational education teachers. Given the time and money constraints, the National Assessment should 
consider how far it can go in including these teachers. 

It is also crucial to remember that the secondary and postsecondary systems probably vary 
widely in terms of preparation, requirements, and certification. The assessment should look at the 
two systems separately and in terms of interaction around articulation. 

Some of the problems can be overcome by using data gathered concurrently from teachers, 
administrators, and students and developing case studies, using the community as the unit of analysis. 

Comments by Panel Chair Gene Bottoms 

In defining an integrated program, it has been assumed that to be a vocational completer in an 
integrated program, a student should have at least three math (two should be Algebra 1 and higher) 
and three science courses (two should be equal to college preparatory chemistry or physics), in 
addition to courses in the area of vocational concentration. When 3,100 transcripts of vocational 
completers were analyzed, findings showed that 29 and 16 percent respectively met the math and 
science requirements, and 12 percent met both. 
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If we integrate what is now being taught, will it make any difference? 
Paul Hill 

The RAND Corporation 
Washington, D.C. 

Effect of performance standards and other measures of accountability on the delivery of 
vocational education services [Sec. 403(b)(7)], 

A study just completed for the NCRVE has resulted in a preliminary report on accountability 
and the Perkins Act implications. The study is an attempt to understand the implications of the 
Perkins Act for state and local performance measurement and accountability. Its other goal is to 
suggest means of implementing Perkins requirements in order to strengthen local accountability 
processes, promote program improvement, and provide quality assurance for top officials such as 
members of Congress. 

Accountability may be defined generically as the relationship between service providers and 
those who have the right to expect to benefit from those services. For vocational education, 
accountability can be seen <& the relationship between employers and providers, between parents or 
students and providers, or the relationship between people who pay for or legislatively authorize the 
programs and the providers. 

Preliminary conclusions of the study are subject to change. However, they show that the 
primary intent of the new Perkins Act was to strengthen state and local accountability, rather than to 
mandate data bases for national reporting. The second conclusion is that the accountability process is 
strong in many localities. This can be seen in employer advisory processes, tracking employer needs 
through student placement, attracting students via job placement success, initiating courses to fill 
emerging needs, and eliminating or upgrading courses with low enrollment and poor placement 
records. 

Where the local accountability process is active and healthy and includes most of the 
stakeholders, the vocational program is substantively tailored to the needs of the labor market it is 
supposed to serve and is responsive to the desires and tastes of people at the local level. This is not 
inconsistent with the Perkins Act accountability requirements. 

However, the accountability process is not strong everywhere. There are a number of 
characteristics of processes that are disconnected from the local labor market or may be serving only 
some part. The standards may be too low. The characteristics include "routine following" mentality 
among vocational education providers; spotty connections with the employer community; lax tracking 
of student placement; low expectations for student skill attainment and job placement; little 
consideration of future labor force needs; and inattention to needs of transient, low income, and 
minority students. 

It is regrettable that the first characteristic of a weak accountability process is the stereotype 
for many people in Washington. They believe that vocational educators are people who are primarily 
interested in maintaining their jobs and making no changes. Everyone knows that local accountability 
processes vary widely, but many fear that the weak processes are prevalent. It is true that even 
processes described as strong may have a bias in that a coalition of employers who are favorable 
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toward the program speak to the state legislature, while the program, in fact, is neglecting the class of 
employers who may be able to offer good opportunities. Local processes are constrained by the local 
businesses* definition of where the economy is going, which might be wrong. There clearly is a role 
for state governments and, to some extent, the federal government to try to stimulate, improve, and 
monitor the quality of the local accountability process. 

A focus on national data bases for accountability can weaken local accountability and may not 
solve the accountability problem at the national level. If the federal government believes that creating 
performance standards, including placement rates, will lead to healthy local processes, it will be in 
trouble. The states will drop their local process monitoring in favor of standard outcomes data 
collection. Furthermore, an upward accountability focus distracts attention from service to local 
students and employers. In addition, universal outcome standards lead to local "gaming" and 
employers* complaints cannot be answered by national averages. Common wisdom is that what is 
measured improves, but this is not necessarily true. At the trivial level, there may be improvement, 
but changes in the underlying dynamic may be negative. Creaming in vocational education is a clear 
example. 

It is hard for many people who have no direct contact with the vocational education system 
and its products to determine its value or distinguish its outcomes from other kinds of education. As 
long as vocational education is inextricably confounded at the local level with other kinds of 
education, trying to show Congress data about vocational education per se may not be the most 
effective way to protect itself. It has some value, but increasing the salience of vocational education 
at the local level so that employers will defend it is probably much more valuable. Thus, the final 
study conclusion is that Perkins Act implementation should lead to direct efforts to strengthen local 
accountability processes. 

Recommendations to the National Assessment. 1) Develop a national picture of local 
accountability processes, including a representative survey of school systems and institutions and 
intensive case studies of a subset. 2) Study state efforts to strengthen links between local providers, 
employers, and parents. 3) Characterize strong versus weak local accountability processes and strong 
versus weak state efforts to promote those processes. 4) Convene a conference to discuss how state 
and federal governments can identify strong and weak local accountability processes, what the federal 
government can do to help states monitor and assess local processes, and how state and federal data 
mandates can support rather than detract from local accountability. 

Potential problems* This is a problem-solving project and is not based on standard methods. 
It is expensive and inherently controversial. 

Discussion 

Major points: 

• Most high schools are probably unaware that they have a problem with academic and 
vocational integration. The local performance accountability focus described is fairly 
traditional. Since studies show that many vocational completers leave high school 
without an adequate academic foundation, what scorekeeping process should be built 
into the National Assessment to promote integration? How can a new context be 
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created in which academic and vocational teachers will come together to solve 
problems? 

Scorekeeping is not the way to solve this problem. A study just completed at RAND 
looked at New York City special purpose and comprehensive public high schools, as 
well as Catholic schools, to see what schools work and do not work for disadvantaged 
students. The Catholic and special purpose high schools (those with a particular focus 
such as vocational education, science, or arts) were alike, and were profoundly 
different from comprehensive public schools. 

The difference between effective and ineffective high schools for poor children is that 
the effective schools are integrated according to a theme. All students are taught all 
classes from a particular point of view. To achieve this kind of integration, the 
school must be built on a set of premises and all vocational and other courses pulled 
together on the basis of those premises. In some cases, there might be no vocational 
education because it might not fit. Integration is a matter of institutional reform, not 
measurement of outcomes. 

But the issue of helping high schools discover that they have a problem should be 
addressed? Is there a data base at the school building level that would help? This 
may be a central issue for the National Assessment. 

The employers, parents, and students themselves should be telling these schools that 
they are not meeting the communities' needs, although the vocational completer's 
initial placement job will probably not provide sufficient feedback for this purpose. 

In the RAND study, schools had ample evidence that they were not working for 
example, a school where no graduate had passed the regent's exam in 5 years and one 
where SO percent of ninth graders did not graduate. In many schools, expectations 
are very low and there is no outside force to cause change. The question of how to 
get effective community pressure to make bad schools good is important, but the 
problem is not one of vocational education, although it may be part of the solution. If 
poor schools and their communities can be helped to see that they do not have to be 
that way, improvement can happen. Funding is also part of the issue, as increased 
funding «can help drive change. 

One criticism of magnet schools is that they work in big cities with high 
concentrations of students in the sending areas, but are more difficult in other areas. 
However, even some rural areas are beginning to think in terms of the community's 
needs and dividing high schools to provide more focused education. A school in 
Moss Point, Mississippi, undertook an effort to provide a well-designed instructional 
system with clear-cut objectives. The student makeup is 60 percent black and 40 
percent white. Over a period of 8 to 9 years, student-achievement levels moved from 
the 10th- 15th to the 65th-70th percentile range. 

The end of this process is a choice system, as high schools built on particular 
principles will mean that people will have reason to go to one rather than another. It 
will be hard to justify expecting students to go to a particular school when another one 
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is better suited to their interests and needs. It also will be hard to justify making 
teachers stay in a school where they are unhappy with the approach and can find one 
they like better. 

The issue of how students come to vocational education has not been discussed. 
Students enter these programs by choice or because they have been coerced in some 
way. However, there also may be a breakdown in the effectiveness of the guidance 
system in relation to vocational education. It might be that the National Assessment 
should look at the guidance element in relation to enrollment. 

Accountability at the postsecondary level is a problem in that students enter these 
institutions for a variety of reasons, and degree or certificate completion may not be 
necessary to their goals. Most are part-time, employed students, and take several 
years to reach their goals. At this level, the only thing that can be counted is 
completions, which do not give much information on accountability in terms of 
program outcomes. 

In postsecondary vocational education, it is difficult to know why people are there 
and, consequently, it is hard to justify the public expenditure. At the community 
college level in Maryland, enrolling students are asked what their goals are, and 
completers are asked whether th^ completed their goals or if die goals changed 
during their time in the college. A problem may arise with grant recipients who must 
be working toward a degree or certificate and who may report false goals. 

In Texas, effort is being made to give academic or general education credit for some 
of the applied technology courses being given in secondary and postsecondary 
institutions. However, teacher preparation has been a barrier, particularly at the 
secondary level. 

This is an important aspect of the linkage between secondary and postsecondary 
institutions. Teacher preparation at the two levels varies to such a great extent that it 
is amazing when they can develop and implement an articulation agreement between 
the two programs. This issue might be one for the National Assessment to consider. 

When we talk about integration, it is important to understand that vocational education 
students often identify with the vocational teachers because of the teachers' 
occupational experience, while they see academic teachers merely as such. This has 
implications for teacher preparation and experience, in that this type of identification 
and bonding helps attract and keep vocational students. One of the values of an 
apprenticeship program is the element of joining and becoming part of an occupational 
community. 

If large numbers of vocational teachers say they do not know how to integrate or 
reinforce subjects such as math and science, how can this gap be bridged in teacher 
preparation? How can academic teachers learn to apply their subjects in a variety of 
contexts? Also, how can we address the problem of vocational education students 
who are not encouraged to read or do homework which requires reading? Reading on 
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the job is different from reading to acquire knowledge for later use. What questions 
can the National Assessment ask about these issues? 

This is a practical issue, as it points to the fact that teachers may not be prepared or 
want to work together toward a common goal. At the school level, leadership and 
vision can help bring teachers together. Even if teachers receive inservice training to 
help them develop the required skills and attitudes, unless the institution has 
leadership committed to effecting positive change and a reward system that supports 
change and creativity, no major change will occur. 

• In some school systems, vocational education is still a "dumping ground." Across the 
educational spectrum, a great deal of money has been directed toward economically 
disadvantaged and disabled students, as it should be. Money has also been directed 
toward higher-achieving students through advanced placement and other programs. 
What will the Perkins Act do for those students in the middle, the average students? 
The Russell Corporation employs many students from this group when they leave high 
school but has not been able to get the school system's cooperation in eliminating 
barriers that would allow average students to be in the vocational track. 

Cathy Stasz 

The RAND Corporation 
Santa Monica, California 

Extent and success of integration of academic and vocational curricula [Sec. 403(b)(10)J. 

This presentation is based on several research studies being done for the NCRVE, including a 
current one by the RAND Corporation which looks at how federal, state, and local policies affect 
implementation of integration policies. The question is whether these policies enable local districts to 
carry out the programs they envision or create barriers to that implementation. 

Integration is a complicated subject and is important to a number of groups. Vocational 
educators are interested in integration because they believe it is a way to respond to criticisms about 
the narrow focus of vocational education and the lack of academic education the vocational students 
receive. Employers are interested because schools are not producing the workers they want, either 
academically or vocationally. Educators and parents are concerned because, in addition to having 
poor reading skills, students do not have the math skills to figure out how many video cassettes they 
can rent with $10.00. And cognitive science researchers, in their efforts to understand how students 
learn and how knowledge can best be imparted to them, strongly believe that lessons can be learned 
from vocational education. 

Vocational educators have used apprenticeship models for a long time, including modeling 
and tutoring as part of their regular teaching practices. Cognitive scientists believe that these and 
other aspects of the hands-on approach used in vocational education could be used in the academic 
classroom to improve students' understanding of math and science. 

When a concept receives attention from so many groups, there is a great deal of variation in 
the kinds of reforms used to implement the concept. With multiple needs and expectations for 
integration, the questions become: What is the purpose of integration? Should we do pervasive skills 
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teaching? Are we interested in improving higher-order thinking skills? Who is the integration 
for—vocational students and teachers? All students? 

There are multiple models of integration. Some are specifically aimed at changing intracourse 
requirements; that is, increasing academic teaching for vocational students. In better models, an 
attempt is made between academic and vocational teachers to work together on courses. There are 
also models where schools are being entirely restructured and the old vocational/academic 
departmentalization has broken down. The schools are organized around occupational clusters and the 
two groups of teachers work together all the time. Classes are blocked so that students can work in 
their vocational area for 3 hours and take their academic courses in shorter blocks. Some of these 
schools have moved toward more student choice and less tracking as students cycle through various 
vocational areas in their freshman and sophomore years and make choices about the occupational 
clusters they will focus on. 

Although this variation has meant that we have yet to develop a firm definition of integration, 
we have identified three elements of the ambitious models of integration. The first element is the 
vertical alignment across courses. In horizontal alignment, academic and vocational teachers 
collaborate, while in vertical alignment, there is a relationship among the courses students take as 
they move through school. In tech-prep, students* courses are related within an occupational cluster 
across the secondary-posts econdary span. 

The second element in the ambitious models is changes in pedagogy. This includes team 
teaching, use of the hands-on approach, and an applied academics curriculum designed to have more 
hands-on work in academic areas for vocational and general track students. The third element is 
organizational change, with clusters, magnet schools, and school-within-a-school arrangements. 
These present a vision of what school might be for all students, and might be the kinds of things the 
new legislation hopes to encourage in terms of restructuring. 

In looking at secondary school issues, state and local policies are considered separately, as 
they are different. Even in states with a clear mandate and guidelines for integration, implementation 
in Ae local districts and schools will vary widely. There are some existing baseline data in the 
NCRVE research on state policies governing integration. Most states are not looking at the more 
ambitious models at this point, but with the advent of the new Perkins legislation, it will be 
interesting to see how the state plans address integration issues. 

Integration does not take place in a vacuum and, in both localities and states, there are 
countervailing policies that can prevent development of certain kinds of innovations. For example, 
California's graduation requirements have resulted in an across-the-board reduction in vocational 
education classes, which are primarily elective in comprehensive schools. Vocational educators have 
had to rethink their classes and recast them to serve the graduation requirements. 

Research issues (secondary level). A state survey should be done to determine current 
policies and methods of interpreting the legislation. About 30 districts could be selected, in either a 
random sample or a purposeful sample which would target possible exemplary models, with both 
comprehensive high schools and vocational/technical schools. In 4-day visits, information would be 
gathered from teachers, students, and community residents to answer such questions as: What is 
integration in this district? What does it mean? What is it trying to achieve? What model or models 
are people putting together to try to reach the goals they have set for integration? What are their 
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plans if they are just getting started and have not done this before? If they have been doing it for a 
while, what problems have they had and how are they trying to cope with those problems? What 
kind of changes is integration going to mean? Is this a reform that will affect both academic and 
vocational students? Is it going to bring teachers closer together in collaboration? Is it going to 
change guidance and counseling? (It is agreed that a knowledge gap exists in regard to this last item.) 

The accountability questions should include: What evidence will indicate that the goals have 
been reached? What means is the local school or district going to use to see if it has reached its 
goals? Will they develop tests? Will they look at completion rates? What will be indicators of 
success? How articulated are those indicators? 

At the postsecondary level, there is a small amount of data. It shows that few states have 
policies for integration at this level. A state survey should be done to determine current policies and 
interpretations of the legislation. One survey by Norton Grubb received about 150 responses from 
community colleges and technical institutes indicating that the most prevalent form of integration is 
requiring vocational students to take general education courses. A few schools are trying to develop 
applied academic curricula or new interdisciplinary courses across academic and vocational lines. 

Research issues (postsecondary level). A survey should be done of a random sample of 
states with a high percentage of Perkins funds going to postsecondary institutions. Within those 
states, locations with a mixture of community colleges, vocational/technical schools, and technical 
institutes should be selected. Instead of an in-depth case study, a telephone survey could be used to 
gather information. Because there is less integration taking place at the postsecondary level, a more 
broad-brush approach could be taken. Programs identified as having an interesting integration 
initiative could receive site visits and be asked basically the same questions as the secondary 
institutions. One question of particular importance is, what is the rationale for integration at the 
postsecondary level? 

Overall research issues and problems. If we are not going to have hard student outcomes 2 
years out, are there some interim indicators that could point to die value of integration? For example, 
if general track students are now taking applied academics and courses they used to take have been 
eliminated, this could be considered a good interim outcome. We will not do a controlled experiment 
to see whether students are learning more than they did in the old courses, but if the educators believe 
these courses and this approach is better and have eliminated the old way, this itself is evidence of an 
integrated approach. 

Even if we could do outcome studies within the time frame, there are technical problems in 
evaluating the consequences of integration. With a variety of reforms ranging from school 
restructuring to small changes in courses, who are the comparison groups? What are the comparable 
measures on student outcomes with such a variety of efforts? Integration is in a developmental stage, 
and it will take a number of years for the successful models to surface, so it is important not to make 
premature judgements. 

It may be possible for the National Assessment to do some focused evaluations. Ohio has 
developed its own applied academics curricula, and some schools are using it. Some are more 
successful than others. It might be very useful to look at some of the comparisons that are naturally 
occurring. It also would be interesting to do an independent evaluation of the popular applied 
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academics curriculum many schools are using. Given the constraints of time and resources, it may be 
the one many schools will ultimately adopt to initiate integration. 

It might also be possible to do a longitudinal study of long-term integration programs that 
have had graduates— for example, Benson Polytechnic in Oregon— which has occupational clusters and 
close communication with the employment community. This and others like it who have recent 
graduates could provide the basis for a longitudinal study that would be in place if the Congress 
should decide to authorize such a study. These programs could also provide information on the 
school-to-work transition and secondary /postsecondary articulation. 

We need to develop a clear vision about what integration can be. We need to identify and 
describe models that schools can emulate and what teachers and administrators need to do to 
implement those models within their local districts. The vocational community can make a major 
contribution by clarifying that vision. The long-term questions for the National Assessment and other 
research projects are: Is integration a good idea? For whom? Does it fulfil the promise it appears to 
offer? Will it eliminate the general track and thus improve education for a large number of students? 

Deborah Strickland 

Virginia Polytechnic Institute and State University 
Blacksburg, Virginia 

Effect of educational reform on vocational education [Sec.403(b)(5)(A)J. 

Virginia Tech is conducting a longitudinal study, funded in part by the NCRVE, to explore or 
map statewide trends in vocational participation vis-a-vis education reform. The original emphasis 
was on high school graduation requirements, which quickly evolved into a broader picture of reform. 
For example, while graduation requirements in some states increased, other states made the existing 
number of requirements more prescriptive. In other states, in addition to the high school graduation 
requirements, postsecondary entrance requirements at 2- and 4-year levels were changed, which had 
an impact on high school students' course choices. In many cases, this precluded their electing 
vocational education courses. 

State secondary school accreditation standards were another reform area impacting vocational 
education in terms of changes in the standards and scope of in-school programs and access to 
programs, both in and outside the school. In some cases, vocational education appeared to benefit by 
a shift from access to programs only in vocational centers to access to certain vocational programs 
within the school. In other cases, standards increased the number of required programs in the 
schools, and vocational education was eliminated. 

Teacher certification is also a factor. Implementation of new certification requirements in 
some states has affected vocational staffing ability, and in some cases, the result is lost courses. 
Equitable funding formulas, while not specifically a reform effort, are having an impact. In poor 
school districts where refunding has been instituted, resources often are obtained for academic 
instruction rather than for vocational education. Parental choice also affects vocational education in 
that schools must market themselves and must determine how and where vocational education fits in. 

A recent study conducted for the Education Commission of the States reviewed reform 
legislation in all 50 states. It identifies some issues in implementation of educational reform policies 
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that have implications for vocational education if it should attempt to initiate its own reform 
strategies: 

• The lack of a clear, coordinated vision for reform within states, resulting in 
inconsistent support and lack of direction. 

• A schizophrenic approach to implementation, resulting in reform activities which are 
more complicated and competitive than they are effective and are difficult to assess in 
terms of student outcomes. 

The issue of leadership is related to the lack of vision, and the lack of coordination among 
reforms sends inconsistent messages to school leaders. There also is a lack of people who know how 
to lead in reform settings. It is hard to see how vocational education could have a positive impact in 
this environment, particularly since it is often seen as being outside the general education reform 
arena. 

There are a number of complications related to vocational education's ability to meet the 
demand for reform. For example, technology education tends to straddle the fence between 
vocational and academic education. In many cases, a program that has been classified as industrial 
arts may change to technology education. Another complication is the division of vocational 
education into a number of various program service areas, which have varying degrees of 
acceptability in terms of being received as alternatives for academic credit in the schools. Leadership 
in vocational education has been receiving increased attention in research efforts. The question is: 
What is the capacity for vocational education to participate in the leadership of education reform 
generally, and in vocational education's own reform efforts? 

Recent research indicates three themes related to vocational education reform to serve all 
students: 1) Providing more context for learning academic subjects than currently exists. 2) 
Providing more opportunities to use knowledge for practical purposes. 3) Providing students a 
broader range of personal, intellectual, and work-related skills than could be provided in traditional 
classroom settings. However, if vocational education is also to continue to provide occupational skills 
to students entering the workforce and to serve the needs of special populations, it has a broad range 
of tasks to consider. 

The issue of delivery of vocational education through postsecondary institutions is another 
level of complication. Although articulation and coordination of secondary and postsecondary 
programs appears to be a critical factor in vocational education reform, in many cases there is little 
coordination and cooperation between them. 

In looking at the impact of reform on vocational education, the Virginia Tech study of 
longitudinal enrollment patterns in vocational education divided states into eight trend categories, four 
showing increasing participation rates and four showing decreasing rates. In well over half the states, 
reforms related to graduation requirements were cited as the one factor with the most negative impact 
on vocational education participation. 

States with positive enrollment rates evidenced the following activities: implementation of 
"2+2", advanced credit, and articulated programs; introduction of exploratory cluster orientation or 
career programs into the middle schools; changes in accreditation standards which required vocational 
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access in schools or area centers; and restructuring vocational education programs and implementing 
more flexible scheduling patterns. And a final factor, particularly important in light of the Perkins 
legislation, was the ability to integrate vocational and academic curricula. 

States in the negative enrollment rate groups cited some specific factors contributing to the 
decrease in vocational participation; where factors cited by the positive groups existed, they were not 
strong. The negative factors cited were emphasis on academics, graduation requirements, and 
accreditation standards to the extent that vocational education was ignored or greatly reduced; 
continued delivery of traditional vocational education, including travel time to centers; increased 
postsecondary entrance requirements; funding caps on personnel or units of vocational instruction; 
lack of coordination with postsecondary institutions and exploratory programs in middle schools; lack 
of integrated curriculum and program innovations; and the image of vocational education. 

From the researcher's viewpoint, it is important to note that decline in area high school 
population or the local economy would also contribute to declining vocational enrollment rates. 

Research issues. The data from the studies cited and various state studies of reform efforts 
provide a data base for looking at the effects of reform on vocational education. NELS 88 and High 
School and Beyond data may also provide some insight into student participation and program 
success. Although most of us appear to be leaning toward qualitative assessment and research, we are 
aware than it is more time consuming than quantitative analyses. Focus groups among state leaders 
within like categories of experience related to reform would be fruitful sources of data, particularly 
regarding identification of models. 

It will also be important for the National Assessment to look as closely at what does not work 
as at what does. Given the complicated and often uncoordinated framework of education reform, we 
may need to ferret out ineffective strategies before progress can be made with the more effective 
approaches. 

Discussion 

Major points: 

• If case studies on integration are done as recommended by Cathy Stasz, it would be 
useful to look at whether or not vocational education students are taught a different 
academic content than college preparatory students, and by different teachers. This 
should also include how expectations for vocational students differ from those for 
other students for example, in the area of homework. However, given the National 
Assessment's resources, it probably is not feasible to do the intensive studies that 
would yield information on teaching practices. It might be possible to get information 
on expectations directly from students, teachers, and administrators. Classroom 
observation can be useful if it is targeted and the researcher has a hypothesis about 
what she or he wants to look at. 

The RAND research suggests that the only thing that makes a difference in education 
is exposing young people to different, higher-level content. If the mentality is that 
vocational students cannot do much and the focus of integration is limited to 
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elementary skills that lead to entry-level jobs, it will not show a difference. 
Expectations for vocational education students are an important variable. 

In relation to the notion of postsecondary integration as simply the presence of general 
education courses in a community college vocational curriculum, this can be 
attributed, in part, to educational reform at the college level. State and accrediting 
agencies are beginning to require these courses across the board. In the move toward 
interdisciplinary courses, applied academic courses tend to be eliminated since they 
cannot be counted toward the general education core. 

In the postsecondary arena, it is important for the institution to think about what 
makes sense for it rather than to try to fit into one of the 8 to 12 existing models. 
The institution needs to make a real effort to understand where integration will make a 
difference for the students in that particular place. With the legislation's emphasis on 
tech-prep as the focus for program improvement, the postsecondary community will 
probably begin to articulate its own vision of integration. 

There is some concern that people might get the idea that all models are equal. It 
would be useful to recognize the need to deal with individual differences and 
perspectives in different places while also developing the idea of a continuum. The 
continuum would convey the message that remediation and teaching students how to 
measure so they can take a drafting course is not only a different model of 
integration, but also is a much less thorough model than many others. Perhaps a 
continuum could be created in terms of degrees of vertical and horizontal integration 
and other factors. 

Part of the problem is that mixing various weak program components will not bring 
about positive effects but could result in further "dumbing down" if academic teachers 
get the idea that vocational students cannot do certain things. 

The other side of the issue of developing a continuum is that it might appear to be 
saying that a particular model exemplifies integration. It is too early in the 
developmental process to say that. 

With regard to classroom observations, there may be some value in asking teachers 
the most important thing they want students to learn and asking students the most 
important thing they learned. There could be some great discrepancies across tracks. 
Looking at teacher-based, school-based, or state-mandated exams might also give us 
an idea of what the school is trying to teach based on what it is measuring and 
rewarding in the exams. 

There is a barrier to some integration and "all aspects of the industry" in certain 
forms of competency-based vocational education. The competency-based concepts are 
excellent, but when they are translated into a large number of specific tasks, they may 
shut out serious integration efforts. 

It is interesting to note that postsecondary health programs tend to be different from 
others and are more integrative in their approach. It might be useful for the National 
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Assessment to look at the health programs and to measure completion rates against 
those of other programs. 

• Every state publishes data on enrollment by specialty, and every state employment 
service publishes occupational and employment data and employment projections. If 
funding is a key point of leverage, would it be going too far if state directors tell the 
people in their states that unless their enrollments have some relationship to 
occupational projections in the state, or even to occupational employment, funding 
will be withheld? If this became part of the accountability process, the question of 
coercion and free choice would come into play. 

This approach could be one way a state education agency could perform a check on 
the health of the local accountability process. It might not go so far as to look for a 
mechanistic connection between the data and punishment, but it could be the basis for 
a dialogue. If the local educators can make the defense that they have a different 
enrollment profile than the state wants because they have a different profile of the 
local labor market, they should not be punished. But if it becomes clear that they are 
not paying attention to the local labor market, the state should intervene. It should 
make an immediate effort to ensure that the locality understands and responds to the 
labor market. This process is used in Utah, where programs are not implemented or 
continued unless they have a direct tie to the local labor market. The state includes 
some programs that are closely tied to market areas where training was needed but 
students did not enroll in the program. 

Final Discussion and Comments 

Major points: 

• It would enrich our discussions and perhaps move our efforts forward, if some 
representatives of the various academic areas were included. This ties in with the idea 
of looking at vocational education in a broader context, which has received heavy 
emphasis in this conference. It also is important to look at academics for research 
reasons, as we need to do comparison and control of all kinds. 

• Statements have been made about giving vocational students instruction in math, 
science, and communication, including advanced courses in the first two subjects. 
Much of the debate here seems to be built on the assumption that we know what 
should be taught on the academic and vocational sides, but our assumptions might not 
be valid. One piece of the equation that is missing is information from the business 
community on what they believe should be taught. 

We might find some dramatic differences if we ask the business community the 
question. For example, the content of high school Algebra I is driven by the needs of 
surveyors' aides when the Northwest Territory was settled. It has not changed since 
then. If we ask, we-will probably find an answer very different from "Algebra F in 
terms of business and industry needs related to math literacy. 
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Most of the business people included in this conference do not represent the smaller 
businesses that employ most of the vocational education graduates. The assessment 
will need to identify a representative sample of those employers who are most directly 
related to the vocational system, both to bring them into the debate and to obtain their 
input for the evaluation. 

• The legislation states that vocational student organizations are to be assessed on the 
degree to which minority students have been involved in the organizations. How will 
that assessment be done? 

This does not pose an assessment problem, in that when the relevant student 
organizations are identified, lists of members can be surveyed for information on 
demographic, racial, and other background characteristics. 

Following a summary by David Boesel of major research questions and issues emerging from 
the conference, presented as the foreword to this report, the meeting was concluded by Milton 
Goldberg, Director, Office of Research, OERI. He noted that the National Assessment has 
considerable importance, not only for vocational education, but also for the broader field of American 
education. He expressed appreciation to the participants for their comments and suggestions, and 
asked that they continue to keep the Office of Research and David Boesel apprised of their own and 
their constituents' concerns. 
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